WEBVTT NOTE This file was generated by Descript 00:00:00.030 --> 00:00:02.940 Judith: Welcome to Berry's In the Interim podcast, where we explore the 00:00:02.940 --> 00:00:06.710 cutting edge of innovative clinical trial design for the pharmaceutical and 00:00:06.710 --> 00:00:09.320 medical industries, and so much more. 00:00:09.990 --> 00:00:10.740 Let's dive in. 00:00:12.425 --> 00:00:13.845 Scott Berry: Well, welcome everybody. 00:00:13.905 --> 00:00:21.175 Welcome to In the Interim, uh, the podcast of, uh, Barry Consultants and 00:00:21.175 --> 00:00:27.435 we, we look at, uh, as much as we can into the science, clinical trial science. 00:00:27.444 --> 00:00:29.304 We look at clinical trial results. 00:00:29.304 --> 00:00:33.675 We talk about statistics, uh, related to medical decision 00:00:33.675 --> 00:00:37.735 making, related to clinical trial results, clinical trial design. 00:00:38.215 --> 00:00:39.305 That's our focus. 00:00:39.780 --> 00:00:45.050 Uh, and so today I'm going to talk about something that turns out 00:00:45.050 --> 00:00:46.670 to be incredibly controversial. 00:00:46.690 --> 00:00:53.900 So the name of this podcast is Politics, Religion, and Ordinal Outcomes. 00:00:55.190 --> 00:00:56.620 Funny name, of course. 00:00:56.640 --> 00:01:02.050 My, my reference of this is that you, you should not at the dinner table, you should 00:01:02.050 --> 00:01:04.430 not talk about politics and religion. 00:01:04.770 --> 00:01:08.110 Turns out you probably shouldn't talk about ordinal outcomes as well. 00:01:08.720 --> 00:01:12.680 Everybody has different reactions to them. 00:01:12.680 --> 00:01:14.740 Everybody analyzes them differently. 00:01:15.110 --> 00:01:17.710 Uh, a passionate feeling about them. 00:01:17.980 --> 00:01:20.910 It's kind of interesting because it's, it's ingrained in the 00:01:20.910 --> 00:01:22.529 history of clinical trials. 00:01:23.715 --> 00:01:27.055 I'm going to essentially describe that almost every endpoint is 00:01:27.055 --> 00:01:29.735 ordinal, so you can't escape this. 00:01:30.705 --> 00:01:32.415 So what are the controversies? 00:01:32.425 --> 00:01:33.515 How do they show up? 00:01:33.715 --> 00:01:35.985 What do I think about ordinal endpoints? 00:01:36.435 --> 00:01:40.165 You may not think that this is a controversial topic. 00:01:40.165 --> 00:01:42.235 It turns out to be really controversial. 00:01:43.095 --> 00:01:47.355 So, ordinal endpoints have been around since the start of clinical trials. 00:01:47.785 --> 00:01:55.795 Uh, I, I, I'm not a, a historian, uh, but my, my scouring of, of the, the, 00:01:56.055 --> 00:02:06.324 of Google and the internet that really the first clinical trial was, uh, 1747. 00:02:06.635 --> 00:02:09.015 And it's attributed to James Lind. 00:02:09.545 --> 00:02:12.405 He was a Scottish naval surgeon. 00:02:13.005 --> 00:02:18.364 He was very, very interested, of course, in 1747 in Scurvy. 00:02:18.365 --> 00:02:19.375 The Scurvy. 00:02:19.865 --> 00:02:25.104 Uh, and he ran what is attributed as the first clinical trial. 00:02:25.845 --> 00:02:31.205 Now, it's not the first randomized clinical trial, and I couldn't 00:02:31.235 --> 00:02:34.654 actually figure out exactly how he assigned patients, and that's the, 00:02:34.664 --> 00:02:36.305 it's an interesting part of the trial. 00:02:36.515 --> 00:02:39.485 So, uh, if you want to read more about this, by the way, 00:02:39.485 --> 00:02:41.095 it's really wonderful reading. 00:02:41.095 --> 00:02:45.145 You'll find yourself in a black hole reading about, uh, James Lind, 00:02:45.695 --> 00:02:50.165 reading about Scurvy, and reading about this particular clinical trial. 00:02:50.935 --> 00:02:55.615 , uh, you can go to james lind library.org, 00:02:55.885 --> 00:02:57.715 uh, and read all about this. 00:02:58.045 --> 00:03:03.115 Uh, so in 1753, he published a treaties on. 00:03:03.775 --> 00:03:05.325 of the scurvy. 00:03:05.715 --> 00:03:08.295 So I'm, I'm, I'm reading parts of this. 00:03:08.565 --> 00:03:10.365 I found it absolutely fascinating. 00:03:10.375 --> 00:03:15.495 And of course, in 1747, he writes that, uh, armies have been 00:03:15.495 --> 00:03:21.395 supposed to have lost more of their men by sickness than by sword. 00:03:22.215 --> 00:03:29.195 And so he makes reference to, uh, that more of the, the English army died from 00:03:29.195 --> 00:03:32.455 scurvy than the French and Spanish armies. 00:03:32.770 --> 00:03:35.280 Uh, themselves, uh, killed. 00:03:35.920 --> 00:03:38.360 And so, huge problem of the time. 00:03:38.690 --> 00:03:45.220 Uh, and he was interested in whether this calamity could be prevented and the 00:03:45.229 --> 00:03:48.720 danger of this destructive evil obviated. 00:03:50.390 --> 00:03:55.410 There were, interestingly at the time, there were tons of, of, of theories 00:03:55.430 --> 00:04:01.445 as to What, what could prevent scurvy, what it was, what was causing it. 00:04:01.455 --> 00:04:03.545 We have a pretty good idea at this time in it. 00:04:04.105 --> 00:04:06.215 Uh, but he was very interested in this. 00:04:06.215 --> 00:04:11.795 On May 20th, 1747, he conducted the first clinical trial. 00:04:12.405 --> 00:04:18.195 He took, as he describes it, 12 patients in the scurvy, on 00:04:18.195 --> 00:04:20.935 board the Salisbury at sea. 00:04:21.295 --> 00:04:24.694 Their cases were as similar as could have them. 00:04:25.565 --> 00:04:32.174 The, they all had putrid gums, the spots, and lassitude, I don't know what that 00:04:32.174 --> 00:04:34.594 means, with weakness of their knees. 00:04:34.604 --> 00:04:38.054 So he of course recognized that we want these patients to look the same. 00:04:38.374 --> 00:04:42.334 He had strange inclusion, exclusion criteria, obviously not written down, 00:04:42.594 --> 00:04:47.114 but he did everything he did, he could to get 12 patients that looked the same. 00:04:48.020 --> 00:04:53.129 He gave them, he describes a common diet, he describes what it is, a water 00:04:53.129 --> 00:04:58.989 gruel sweetened by sugar in the morning, fresh mutton broth for dinner, and of 00:04:58.989 --> 00:05:03.399 course he refers to supper as different than dinner, barley with raisins, 00:05:03.400 --> 00:05:06.669 rice and currants, and with wine. 00:05:07.080 --> 00:05:08.869 Uh, so he, he, he, he. 00:05:09.020 --> 00:05:13.790 did really smart things for the first clinical trial, trying to get patients 00:05:13.790 --> 00:05:19.060 to look the same, tried to treat them the same in things except for the 00:05:19.060 --> 00:05:21.039 interventions he was interested in. 00:05:21.660 --> 00:05:26.380 So he had 12 patients and he had six different treatments. 00:05:27.350 --> 00:05:30.200 He gave two of them court of cider. 00:05:30.520 --> 00:05:31.119 a day. 00:05:31.929 --> 00:05:38.999 He gave two of them 25 guts of elixir vitriol three times a day. 00:05:39.919 --> 00:05:44.739 He gave two of them spoonfuls of vinegar three times a day. 00:05:46.239 --> 00:05:50.760 Two of them, two, and he describes it interestingly, two of the worst 00:05:50.770 --> 00:05:58.549 patients, their tendons in the ham, rigid seawater, a half a pint a day. 00:05:59.875 --> 00:06:05.565 He gave two of them, two oranges and one lemon per day. 00:06:06.565 --> 00:06:12.114 And two of them, bigness of nutmeg, three times a day. 00:06:12.285 --> 00:06:15.014 This was a surgeon recommended therapy. 00:06:15.015 --> 00:06:20.345 Again, lots of people were describing what they would do for the treatment of scurvy. 00:06:21.295 --> 00:06:25.975 Uh, by the way, that, that, I don't know what that is, but apparently it had 00:06:25.985 --> 00:06:34.595 garlic, mustard seed, radish, balsam of Peru, uh, gummer, and cream of tartar. 00:06:35.795 --> 00:06:39.075 Uh, and the interesting thing in all of this is he does not 00:06:39.085 --> 00:06:41.015 describe how they were assigned. 00:06:41.595 --> 00:06:46.615 They were randomized, he assigned them, uh, an interesting part of a 00:06:46.615 --> 00:06:49.895 clinical trial that we now think a really important part of the trial. 00:06:50.304 --> 00:06:53.544 Uh, and that's really left out of the description, which is a 00:06:53.544 --> 00:06:56.994 fascinating aspect of this 1747 trial. 00:06:57.234 --> 00:07:02.054 I'm more, much more interested in today's discussion of ordinal endpoints, of how 00:07:02.064 --> 00:07:05.004 he classified the outcomes of this trial. 00:07:06.134 --> 00:07:12.450 So he writes, Uh, and I'm quoting from his writing, the consequence was the 00:07:12.450 --> 00:07:18.419 results, the most sudden and visible good effects were perceived from the use of 00:07:18.419 --> 00:07:27.429 the oranges and lemons one patient taken being at the end of six days fit for duty. 00:07:28.630 --> 00:07:32.710 The spots were not indeed quite off his body, nor his gum sound. 00:07:33.060 --> 00:07:38.049 The other was the best recovered of any of his condition. 00:07:38.739 --> 00:07:43.150 And being deemed well was appointed nurse of the rest of the sick. 00:07:43.510 --> 00:07:46.620 So, I don't think this happens in current clinical trials. 00:07:46.620 --> 00:07:50.310 that A patient did so well that they become a caregiver of the other 00:07:50.310 --> 00:07:51.700 patients in the clinical trial. 00:07:52.230 --> 00:07:54.270 Um, uh, in that scenario. 00:07:54.905 --> 00:07:59.405 But the description of this is the two best patients out of the 12 were 00:07:59.405 --> 00:08:01.284 those that were on a single treatment. 00:08:01.645 --> 00:08:03.304 The oranges and lemon. 00:08:03.914 --> 00:08:05.025 In this scenario. 00:08:05.274 --> 00:08:08.034 So his outcome was an ordinal outcome. 00:08:08.064 --> 00:08:12.734 Actually resembles what we would today, we might call a door patient. 00:08:12.744 --> 00:08:16.465 That are almost a win ratio type thing. 00:08:16.765 --> 00:08:18.825 That comparing patients to patients. 00:08:19.155 --> 00:08:20.835 These patients did the best. 00:08:21.734 --> 00:08:27.585 There's no quantitative aspect that this patient was five times better than 00:08:27.585 --> 00:08:29.784 that patient or double that patient. 00:08:29.794 --> 00:08:32.895 There wasn't a mean time to recovery and all of this. 00:08:33.164 --> 00:08:34.864 It was an ordinal outcome. 00:08:35.364 --> 00:08:41.664 By the way, if you calculate a p value for the probability that lemons 00:08:41.664 --> 00:08:48.735 and oranges would have the two best patients, it's one out of 66 or 015. 00:08:50.010 --> 00:08:53.290 You, of course, would say, well, wait a minute, there's multiplicities here. 00:08:53.290 --> 00:08:55.130 There were six different treatments. 00:08:55.579 --> 00:09:00.130 What's the probability that any one of them would have had the two best? 00:09:00.400 --> 00:09:05.319 Well, that's one out of eleven, uh, in that, which is 091. 00:09:05.339 --> 00:09:08.540 So, adjusting for multiplicities, you get 091. 00:09:10.360 --> 00:09:12.410 I'm more interested in the, the endpoint. 00:09:12.439 --> 00:09:16.129 Now, interestingly, in part of this, and this may resonate with people, 00:09:16.129 --> 00:09:18.920 that, This didn't change anything. 00:09:18.920 --> 00:09:20.949 We now know this was actually the right treatment. 00:09:21.360 --> 00:09:27.240 that it's about vitamin C and lemons and oranges would have prevented the scurvy 00:09:27.240 --> 00:09:29.199 and would have had a massive impact. 00:09:29.679 --> 00:09:34.949 Uh, it took 47 years beyond this before this became recognized 00:09:34.949 --> 00:09:36.050 as a way to treat patients. 00:09:36.545 --> 00:09:41.855 Interestingly, the thought was that you give dehydrated, uh, fruit because 00:09:41.855 --> 00:09:45.775 it was hard to keep fresh fruit on, on board, uh, ships and all that. 00:09:46.065 --> 00:09:50.475 And it turns out that loses much of the vitamin C content and it's not clear 00:09:50.475 --> 00:09:52.095 that would have had any actually benefit. 00:09:52.765 --> 00:10:00.215 So, CMC issues and PKPD issues could have really helped James Lind in 1747. 00:10:00.335 --> 00:10:06.245 So, that was the first clinical trial, and again, you should go read all about this. 00:10:06.255 --> 00:10:07.725 It's absolutely fascinating. 00:10:08.504 --> 00:10:13.095 Um, the first randomized human clinical trial is attributed 00:10:13.105 --> 00:10:14.625 to Austin Bradford Hill. 00:10:15.685 --> 00:10:21.204 And he the, he published a paper in 1948 of the first randomized clinical 00:10:21.204 --> 00:10:27.055 trial, which was ex experimenting on streptomycin for the treatment 00:10:27.084 --> 00:10:30.354 of pulmonary tur tuberculosis. 00:10:30.685 --> 00:10:32.604 And this was published in the. 00:10:33.814 --> 00:10:36.375 British Medical Journal in 1948. 00:10:38.515 --> 00:10:45.305 This trial enrolled 107 patients and the difference between James Lynn's, James 00:10:45.474 --> 00:10:50.194 Lynn's trial and this one was that the patients were randomized to the treatment. 00:10:50.954 --> 00:10:54.614 And they were randomized to the control group or they were 00:10:54.614 --> 00:10:58.864 randomized to streptomycin in it. 00:10:58.930 --> 00:11:02.729 Uh, the publication of this is also very interesting. 00:11:02.729 --> 00:11:04.170 There's a traditional table one. 00:11:04.170 --> 00:11:07.319 Maybe he invented the table one and the table two. 00:11:07.620 --> 00:11:11.779 Uh, but table one give it, gave demographics of the trial and 00:11:11.779 --> 00:11:13.680 table two gave the results. 00:11:14.000 --> 00:11:19.400 So what was the end point of this first randomized clinical trial of humans? 00:11:19.420 --> 00:11:24.069 There were randomized, uh, uh, uh, people probably understand there were randomized. 00:11:24.475 --> 00:11:27.865 uh, agricultural experiments, but there were not randomized 00:11:27.865 --> 00:11:29.775 human experiments before this. 00:11:30.235 --> 00:11:35.774 Um, the primary endpoint of this was a more, what you more think of now as 00:11:35.775 --> 00:11:37.904 a more traditional ordinal endpoint. 00:11:37.905 --> 00:11:41.974 It was a six level ordinal endpoint. 00:11:41.974 --> 00:11:45.365 The top level, but was considerable improvement. 00:11:45.955 --> 00:11:52.135 The next was moderate or slight improvement, then no material change. 00:11:52.834 --> 00:11:59.535 Moderate or slight deterioration, considerable deterioration, and death. 00:12:00.204 --> 00:12:02.064 Six, six outcomes. 00:12:02.084 --> 00:12:04.374 By the way, they aren't given numbers. 00:12:05.420 --> 00:12:10.800 They aren't given, you could consider giving them letters, A, B, C, D, E, F. 00:12:10.920 --> 00:12:12.730 F is the worst, A is the best. 00:12:13.120 --> 00:12:17.350 So all this is, is, is, it's ordinal in that one is better than 00:12:17.350 --> 00:12:23.399 the other one, but no numeric, uh, number is given as how much better. 00:12:24.030 --> 00:12:28.260 We think of these as ordinal outcomes, and it's probably what 00:12:28.260 --> 00:12:29.990 you know as ordinal outcomes. 00:12:30.800 --> 00:12:33.689 So controversially, how did he analyze this? 00:12:33.709 --> 00:12:38.600 We would fight today, I mean fight in a good way, that we would fight 00:12:38.600 --> 00:12:40.079 about how to analyze this endpoint. 00:12:40.769 --> 00:12:41.530 What do you do? 00:12:41.670 --> 00:12:44.140 He analyzed it actually in multiple ways. 00:12:44.735 --> 00:12:50.975 He, um, he talked about, uh, in this outcome, the first thing he 00:12:50.975 --> 00:12:56.224 did was he talked about deaths and deaths were four, I'm sorry, 7. 00:12:56.224 --> 00:13:03.264 3 percent in streptomycin and it was 27 percent in control. 00:13:03.685 --> 00:13:06.965 And he talked about this being less than one in a hundred 00:13:07.225 --> 00:13:08.695 chance that this would happen. 00:13:08.825 --> 00:13:10.795 He then said the best. 00:13:12.985 --> 00:13:18.525 If you split it in any particular way, the best split for streptomycin for 00:13:18.525 --> 00:13:25.665 this was considerable improvement, which was 51 percent to 8%. 00:13:26.265 --> 00:13:29.404 One in a million, he talks about the likelihood that 00:13:29.404 --> 00:13:30.765 that would happen by chance. 00:13:31.005 --> 00:13:33.195 So he's giving p values, he's analyzing it. 00:13:33.195 --> 00:13:36.525 We would recognize, ooh, the multiplicity of looking at the 00:13:36.525 --> 00:13:38.715 best place and what does that mean? 00:13:38.715 --> 00:13:40.665 And that gets everybody in a bob. 00:13:41.189 --> 00:13:47.569 Um, now this was an easy analysis where any way you analyze this endpoint, 00:13:47.589 --> 00:13:49.219 it looked really, really good. 00:13:49.770 --> 00:13:52.689 Uh, and so there's not going to be a whole lot of struggle. 00:13:53.030 --> 00:13:58.570 Um, uh, on this end point in there, and I went back and took the data and I 00:13:58.590 --> 00:14:03.910 fit a proportional odds model to this, and you get a proportional odds of 5. 00:14:03.960 --> 00:14:07.700 43 to the good of streptomycin. 00:14:08.089 --> 00:14:12.040 Uh, the lower bound of the 95 percent confidence interval is 2. 00:14:12.040 --> 00:14:13.259 64. 00:14:13.260 --> 00:14:15.100 This is a massive effect. 00:14:15.540 --> 00:14:18.139 P value of that is 6. 00:14:18.139 --> 00:14:19.510 3 times 10 to the minus 6. 00:14:20.545 --> 00:14:21.525 Awesome data. 00:14:21.785 --> 00:14:23.655 Uh, makes it really easy. 00:14:23.655 --> 00:14:27.175 Everybody can, you know, look at this and say, this is clinically 00:14:27.175 --> 00:14:28.344 really, really important. 00:14:28.785 --> 00:14:29.555 Um, in it. 00:14:30.395 --> 00:14:35.115 That was, um, that was some 75 years ago. 00:14:35.634 --> 00:14:40.675 We still fight about how to analyze orbital endpoints, um, within this. 00:14:41.655 --> 00:14:43.635 Um, okay. 00:14:44.165 --> 00:14:47.585 So we're still fighting about this. 00:14:48.015 --> 00:14:52.504 I'm going to focus to talk more about where are we today, what are we doing in 00:14:52.505 --> 00:14:57.754 Ordinal Endpoints, what do I think about Ordinal Endpoints, uh, from this, as I, 00:14:57.754 --> 00:14:59.495 I'm going to pick a particular example. 00:14:59.504 --> 00:15:06.755 Very similar to, uh, uh, Austin Bradford Hills, Bradford Hills, uh, endpoint, 00:15:07.165 --> 00:15:10.480 uh, of, A six level ordinal endpoint. 00:15:10.490 --> 00:15:12.830 We're going to talk about the modified Rankin score. 00:15:13.330 --> 00:15:19.029 The modified Rankin score is a seven level ordinal outcome. 00:15:19.370 --> 00:15:23.600 It's used on the neurological status of patients. 00:15:24.170 --> 00:15:27.359 It's quite common in a number of scenarios. 00:15:27.360 --> 00:15:32.830 It's quite common in stroke trials, where the endpoint is your modified 00:15:32.840 --> 00:15:36.370 Rankin status at 90 days, 180 days. 00:15:36.615 --> 00:15:37.564 different points. 00:15:37.865 --> 00:15:41.045 Um, the endpoint itself is seven levels. 00:15:41.334 --> 00:15:44.125 They do give numeric values to these. 00:15:44.935 --> 00:15:49.175 The numeric values that now we'll talk about, are they relevant or not? 00:15:49.204 --> 00:15:50.784 Are they just labels? 00:15:51.884 --> 00:15:52.974 And they're labels. 00:15:53.015 --> 00:15:59.554 You could have labeled them, uh, A, B, C, D, E, F, G. 00:16:02.855 --> 00:16:05.425 But I'll give the labels that are there, you may be familiar with them. 00:16:05.425 --> 00:16:06.865 Zero is no symptoms. 00:16:06.925 --> 00:16:08.385 Perfect neurological stat. 00:16:09.305 --> 00:16:12.305 One is no significant disability. 00:16:12.505 --> 00:16:15.254 You're able to carry out usual activities. 00:16:15.595 --> 00:16:17.185 Uh, zero is better than one. 00:16:18.435 --> 00:16:20.224 We can argue about how much better. 00:16:20.765 --> 00:16:21.725 We're going to get to that. 00:16:22.165 --> 00:16:24.165 Two is slight disability. 00:16:24.415 --> 00:16:28.074 You can look after your own affairs, but unable to carry 00:16:28.074 --> 00:16:30.175 out all previous activities. 00:16:30.314 --> 00:16:32.035 So you have some disability. 00:16:32.515 --> 00:16:34.515 Uh, one is better than two. 00:16:34.845 --> 00:16:35.895 Zero is better than one. 00:16:35.895 --> 00:16:36.964 One is better than two. 00:16:37.694 --> 00:16:40.185 Then three is moderate disability. 00:16:40.185 --> 00:16:41.185 You need help. 00:16:41.785 --> 00:16:43.035 You require help. 00:16:44.140 --> 00:16:50.339 Four, moderate to severe disability, unable to attend to your own 00:16:50.349 --> 00:16:52.589 bodily needs without assistance. 00:16:52.959 --> 00:16:58.329 Again, a higher level of support needed to carry out daily activities. 00:16:59.099 --> 00:17:05.364 Five is severe disability, bedridden and requiring constant nursing care. 00:17:06.345 --> 00:17:09.755 It's commonly given the label of vegetative state. 00:17:10.155 --> 00:17:16.075 Um, uh, clearly a, a worse state than four, worse than zero through three. 00:17:16.514 --> 00:17:17.895 And then six is dead. 00:17:18.414 --> 00:17:20.054 Uh, no neurological status. 00:17:20.704 --> 00:17:22.714 So, these are the seven outcomes. 00:17:23.014 --> 00:17:26.744 It's a very, very common outcome in clinical trials. 00:17:27.185 --> 00:17:29.455 The question is, how do we analyze this thing? 00:17:29.465 --> 00:17:34.435 How do we analyze those seven outcomes in a clinical trial? 00:17:34.774 --> 00:17:38.485 There's huge disagreement that goes on about, uh, analyzing this. 00:17:38.824 --> 00:17:42.334 There's disagreement about the medical interpretation of these. 00:17:42.345 --> 00:17:47.135 There's disagreement about statistically how to analyze them, uh, the relative 00:17:47.135 --> 00:17:49.365 meaning of the values of them. 00:17:50.254 --> 00:17:52.554 Uh, and different analyses are used. 00:17:52.555 --> 00:18:00.435 The, the, the first way to potentially analyze these, and I shudder to say 00:18:00.435 --> 00:18:03.104 it, but is to dichotomize these. 00:18:04.065 --> 00:18:08.774 Very common in clinical trials, the modified Rankin is dichotomized. 00:18:08.814 --> 00:18:11.825 There's even a name to the dichotomization. 00:18:12.325 --> 00:18:14.915 Uh, and it's disabled or not. 00:18:15.935 --> 00:18:17.385 So 0 through 2. 00:18:18.329 --> 00:18:21.689 is considered a responder, a good outcome. 00:18:22.239 --> 00:18:25.589 3 through 6 is considered a bad outcome. 00:18:26.089 --> 00:18:30.729 And the clinical trial analyzes how many of those are 0 to 2 and 00:18:30.729 --> 00:18:33.059 how many of those are 3 to 6. 00:18:33.589 --> 00:18:37.500 That's a way to numerically analyze these outcomes. 00:18:38.620 --> 00:18:42.849 Now I'm going to walk through this a little bit and I want to start by saying 00:18:42.859 --> 00:18:45.219 this is mathematically equivalent. 00:18:46.300 --> 00:18:56.069 to giving a weight of 0, 1, and 2, a value of 1, giving 3 through 6 a value of 0. 00:18:56.990 --> 00:19:03.290 I could say that's my utility, and analyze that way, and I get 00:19:03.290 --> 00:19:06.399 mathematically the exact same answer. 00:19:08.990 --> 00:19:11.080 Okay, so. 00:19:11.710 --> 00:19:17.440 It is a weighting by doing that dichotomous and you're saying having 00:19:17.440 --> 00:19:20.720 a zero is equivalent to having a two. 00:19:22.109 --> 00:19:25.410 Having a three is equivalent to dead. 00:19:26.109 --> 00:19:31.089 Absolutely, that is the assumption that's being made when you 00:19:31.099 --> 00:19:33.020 analyze the trial that way. 00:19:34.430 --> 00:19:36.770 I shudder at that. 00:19:37.020 --> 00:19:38.820 I shudder at that clinically. 00:19:39.290 --> 00:19:42.410 I shudder at it statistically for a couple reasons. 00:19:42.879 --> 00:19:49.699 The power of a trial that lumps everybody together, 0-2 3-6 is lower 00:19:50.090 --> 00:19:51.929 than a different way to analyze it. 00:19:51.949 --> 00:19:54.360 I think it's clinically harder to interpret. 00:19:54.610 --> 00:20:01.250 I don't think any person has a utility function that is equal to that. 00:20:01.740 --> 00:20:03.579 But it's very commonly done. 00:20:05.155 --> 00:20:09.335 Okay, another way to analyze this endpoint, you, you, you can 00:20:09.335 --> 00:20:13.375 probably understand what I think about, uh, zero to two or any 00:20:13.385 --> 00:20:15.725 dichotomization of that endpoint. 00:20:17.555 --> 00:20:20.184 What about a proportional odds model? 00:20:20.524 --> 00:20:26.685 So a proportional odds model is a statistical assumption that the effect 00:20:27.295 --> 00:20:30.665 of moving patients to a higher level. 00:20:31.645 --> 00:20:37.034 is, has a statistical, uh, odds of that happening. 00:20:37.645 --> 00:20:43.054 We assume it's the same of moving people from 6 to better than 6, 00:20:43.885 --> 00:20:50.675 from 5 and 6 to better than 5 and 6, from 6 to better than 6. 00:20:50.685 --> 00:20:54.594 The odds of that is constant across the scale. 00:20:56.570 --> 00:20:58.910 There's a strong, I don't want to say strong. 00:20:59.320 --> 00:21:00.710 I'm sure I shouldn't say that word. 00:21:00.750 --> 00:21:03.760 There's a statistical assumption that comes with that. 00:21:04.359 --> 00:21:07.689 That, that odds is the same across the scale. 00:21:08.604 --> 00:21:11.485 Now that gets statisticians all ruffled. 00:21:11.854 --> 00:21:16.284 It's almost like when you go to graduate school, you have to 00:21:16.405 --> 00:21:20.385 question that assumption in any model, or you're not worth your 00:21:20.385 --> 00:21:23.804 statistical salt, uh, uh, within this. 00:21:24.084 --> 00:21:26.294 It is a, it is an assumption. 00:21:26.425 --> 00:21:29.455 Now, and it can be violated. 00:21:30.410 --> 00:21:35.650 The interesting thing about it is that, that way to analyze 00:21:35.650 --> 00:21:40.499 that ordinal endpoint imposes a utility on each of those outcomes. 00:21:41.619 --> 00:21:45.259 Remember, dichotomizing them imposes a utility. 00:21:46.870 --> 00:21:48.520 And you can mathematically write it down. 00:21:50.160 --> 00:21:54.160 A proportional odds model imposes a utility on it and you can 00:21:54.160 --> 00:21:55.480 mathematically write it down. 00:21:56.265 --> 00:21:58.075 What is the utility? 00:21:58.605 --> 00:22:02.055 The interesting thing is the utility of that is based on 00:22:02.055 --> 00:22:03.995 the prevalence of the outcomes. 00:22:05.275 --> 00:22:12.344 If you have a lot of 0s and 1s and 10 percent deaths but 90 percent 0s and 00:22:12.344 --> 00:22:18.855 1s, your utility you're imposing is highly weighting 0 to 1 differences. 00:22:19.525 --> 00:22:21.894 Because that's where your prevalence is. 00:22:22.754 --> 00:22:28.615 Death, and above that, is down weighted because they aren't very prevalent. 00:22:29.334 --> 00:22:31.064 You can mathematically write it down. 00:22:31.064 --> 00:22:34.665 I could give you the utility functions as you're imposing that upon your input. 00:22:36.114 --> 00:22:37.725 People don't think about it that way. 00:22:37.725 --> 00:22:39.185 It's a little bit uncomfortable. 00:22:39.625 --> 00:22:45.225 If, alternatively, you have a lot of 4's, 5's, and 6's and very few 0's and 1's, 00:22:45.524 --> 00:22:49.484 but you have some, you're down weighting those because they're not prevalent. 00:22:51.315 --> 00:22:56.815 You are imposing a mathematical weight of those outcomes by analyzing that way. 00:22:56.815 --> 00:22:58.045 You can't get out of that. 00:22:59.965 --> 00:23:00.605 Okay. 00:23:01.184 --> 00:23:02.744 Uh, I'm not saying that's bad. 00:23:02.825 --> 00:23:03.775 I've used it. 00:23:04.075 --> 00:23:06.324 By the way, I've used both of those in clinical trials. 00:23:06.684 --> 00:23:10.545 Um, I struggle with the dichotomization, but I've done it. 00:23:10.935 --> 00:23:16.074 Um, I've done proportional odds model, and I don't I don't 00:23:16.074 --> 00:23:17.405 necessarily shudder at them. 00:23:17.754 --> 00:23:22.254 Um, but I, I understand I'm imposing this strange sort of weight on them. 00:23:23.415 --> 00:23:23.915 Okay. 00:23:25.195 --> 00:23:27.885 There, there are other ways to analyze these endpoints. 00:23:27.914 --> 00:23:29.534 We could do a Wilcoxon test. 00:23:29.534 --> 00:23:31.604 We could do a Cox model on them. 00:23:32.235 --> 00:23:36.745 Um, first of all, I want to point out that there are tons of ordinal outcomes. 00:23:37.635 --> 00:23:40.854 This is the one a lot of people are familiar with. 00:23:41.460 --> 00:23:45.800 Um, mortality is a ordinal outcome. 00:23:46.060 --> 00:23:47.700 Yes, no, mortality. 00:23:47.940 --> 00:23:48.480 It's ordinal. 00:23:48.740 --> 00:23:50.779 Now, that's a pretty simple dichotomous one. 00:23:50.780 --> 00:23:51.640 It's binary. 00:23:52.710 --> 00:23:58.279 Time to death, overall survival in a cancer clinical trial, or any 00:23:58.279 --> 00:24:00.540 clinical trial, is an ordinal outcome. 00:24:01.830 --> 00:24:06.440 A year is better than six months, but you don't say how much. 00:24:06.960 --> 00:24:10.440 You just say it's better when you run a Cox proportional 00:24:10.450 --> 00:24:12.780 hazards model or a log rank test. 00:24:13.050 --> 00:24:20.100 You're analyzing an ordinal outcome On that when you're analyzing infarct 00:24:20.129 --> 00:24:26.120 volume or tumor size That's we sometimes people say that's quantitative. 00:24:26.540 --> 00:24:28.030 Well, no, it's ordinal 00:24:30.554 --> 00:24:35.304 You're imposing upon it the difference between them of one 00:24:35.304 --> 00:24:38.254 unit being a numerical difference. 00:24:38.274 --> 00:24:43.185 You're imposing upon a quantitative weight of that ordinal scale, but it's ordinal. 00:24:44.175 --> 00:24:45.695 Almost everything is ordinal. 00:24:46.294 --> 00:24:51.215 The best corrected visual acuity where you're looking at a chart and saying how 00:24:51.215 --> 00:24:53.034 many letters can I see, it's ordinal. 00:24:53.804 --> 00:24:55.484 We calculate a mean of them. 00:24:56.235 --> 00:25:01.075 We make the assumption that one letter is better, no matter where you are in the 00:25:01.075 --> 00:25:03.645 scale, that it has a constant difference. 00:25:04.034 --> 00:25:06.635 We're imposing a quantitative weight on them. 00:25:07.195 --> 00:25:08.705 People are comfortable with that. 00:25:09.995 --> 00:25:16.815 When I impose a numerical value to the, you, to the, uh, modified 00:25:16.825 --> 00:25:19.784 Rankine, people jump up in arms. 00:25:19.865 --> 00:25:20.345 People. 00:25:21.095 --> 00:25:23.280 Scientists regulators. 00:25:23.719 --> 00:25:28.240 So there have been multiple, there have been several studies that look at patient 00:25:28.240 --> 00:25:36.504 preference for the modified Rankin, that look at, um, the economic value of 00:25:36.504 --> 00:25:38.364 them and they're actually very similar. 00:25:38.814 --> 00:25:42.854 We have taken them and said, okay, we're not going to do a proportional odds model. 00:25:42.854 --> 00:25:45.024 We're not going to do dichotomous. 00:25:45.495 --> 00:25:47.574 Um, nobody has that weight. 00:25:47.834 --> 00:25:51.264 We're going to use this and we're going to call it a utility function. 00:25:51.274 --> 00:25:56.314 By that weight, by the way, the weight that is used on that is a full weight of 00:25:56.314 --> 00:26:03.260 1 if you're a 0, 91 if you're a 1, 76 0. 00:26:03.260 --> 00:26:05.070 65, 0. 00:26:05.199 --> 00:26:08.980 33, and then 0 for both 5 and 6. 00:26:09.010 --> 00:26:10.919 That those are equally bad. 00:26:12.490 --> 00:26:13.820 That's a utility function. 00:26:13.820 --> 00:26:15.070 You can analyze it. 00:26:15.280 --> 00:26:17.919 Very simple statistical models. 00:26:17.919 --> 00:26:19.959 No assumption of proportional odds. 00:26:20.250 --> 00:26:21.760 And we can analyze it that way. 00:26:22.720 --> 00:26:26.500 Uh, now, people get frustrated by that. 00:26:26.975 --> 00:26:31.184 Lots of people in terms of regulators, what does this mean, 00:26:31.184 --> 00:26:35.514 and the first criticism is not everybody has that utility weight. 00:26:35.774 --> 00:26:39.794 You're using two different studies, but how do we know the people that 00:26:39.794 --> 00:26:41.475 you're analyzing that have that weight? 00:26:43.004 --> 00:26:48.905 The problem is any way you analyze this endpoint has that same issue. 00:26:49.014 --> 00:26:52.884 If you dichotomize them, you're imposing that everybody in your 00:26:52.884 --> 00:26:55.324 study has that weight of 111000. 00:26:57.634 --> 00:26:59.905 My claim is nobody has that weight. 00:27:00.705 --> 00:27:03.835 But yet, it's very commonly done. 00:27:04.365 --> 00:27:07.189 But nobody's frustrated by that. 00:27:09.060 --> 00:27:11.770 Bothers me because I don't think anybody has that weight. 00:27:12.550 --> 00:27:19.340 P, studies, phase 3 clinical trials at FDA use proportional odds model. 00:27:20.879 --> 00:27:22.050 There's a comfort in that. 00:27:22.060 --> 00:27:25.179 You got to check assumptions, you got to look at proportional odds. 00:27:25.659 --> 00:27:28.759 The usual thing we do for Cox models and all that. 00:27:29.329 --> 00:27:30.739 You're imposing a weight. 00:27:30.749 --> 00:27:33.409 How do we know the people in your study have that weight? 00:27:35.415 --> 00:27:40.064 My point is it's a false criticism because you cannot escape it. 00:27:40.675 --> 00:27:43.465 You have to weight the endpoints any way you analyze them. 00:27:44.504 --> 00:27:48.935 All of these endpoints we impose a weight on them when we go into a clinical trial. 00:27:49.955 --> 00:27:55.225 But when we specify it up front and try to justify that this is a good 00:27:55.225 --> 00:27:59.875 weighting system, everybody throws arrows at it and, oh, that's bad. 00:28:01.425 --> 00:28:05.585 But dichotomous and proportional odds, that's a good way to do it. 00:28:07.525 --> 00:28:11.215 Alright, so what do I think of this? 00:28:11.415 --> 00:28:14.455 I think it's the really hard thing in a clinical trial. 00:28:14.885 --> 00:28:18.855 It's the way as we move forward we're going to get more and more fine 00:28:18.864 --> 00:28:20.565 grained understanding of endpoints. 00:28:21.185 --> 00:28:24.375 Our endpoints are going to be more strength, more valuable. 00:28:25.385 --> 00:28:27.014 We have to do this. 00:28:27.095 --> 00:28:28.125 It's hard. 00:28:28.820 --> 00:28:30.180 Science is hard. 00:28:30.190 --> 00:28:31.620 Medicine is hard. 00:28:32.120 --> 00:28:37.270 Hiding behind ad hoc ways to do this, I think, just leads us to bad places. 00:28:37.739 --> 00:28:42.830 It disconnects the statistics from the clinical outcomes, from 00:28:42.830 --> 00:28:44.769 the doctors, from the clinicians. 00:28:45.109 --> 00:28:47.090 This is where we need to go. 00:28:47.470 --> 00:28:48.750 We need baby steps. 00:28:48.750 --> 00:28:50.060 We need to work there. 00:28:50.310 --> 00:28:54.170 We need to show it's valuable, that it's good for regulators, that it's good for 00:28:54.170 --> 00:28:56.170 patients, it's good for statisticians. 00:28:56.645 --> 00:28:59.955 But it's really the only way to do this. 00:29:00.285 --> 00:29:02.685 So I think as our endpoints become more. 00:29:03.020 --> 00:29:11.810 More, um, uh, less signal, uh, more signal, less noise, wearables, uh, daily 00:29:11.810 --> 00:29:17.070 values, our iPhone, our iWatches, our, our things, calculating things for us. 00:29:17.070 --> 00:29:18.609 These are all ordinal outcomes. 00:29:19.489 --> 00:29:21.594 That this is the only way forward. 00:29:21.815 --> 00:29:23.185 We need to go there. 00:29:23.485 --> 00:29:24.655 Let's do baby steps. 00:29:24.655 --> 00:29:29.075 Let's analyze the modified Rankine in really, really better ways, smarter 00:29:29.075 --> 00:29:33.295 ways, explicit ways, uh, in this. 00:29:34.715 --> 00:29:39.795 And please, please, let's not just dichotomize really nice endpoints. 00:29:40.215 --> 00:29:44.584 Um, uh, again, I don't think anybody has that weight system. 00:29:45.225 --> 00:29:49.345 Alright, well I hope, I hope I didn't defend you like I would if we 00:29:49.345 --> 00:29:51.415 talked about religion or politics. 00:29:51.765 --> 00:29:54.895 It's a controversial thing, but I think it's a really important 00:29:54.895 --> 00:29:56.055 thing in clinical trials. 00:29:56.435 --> 00:29:59.345 I hope you're, you, you, you enjoyed the discussion. 00:29:59.915 --> 00:30:03.165 And until the next time, we are in the interim. 00:30:03.355 --> 00:30:03.375 Yeah.