WEBVTT NOTE This file was generated by Descript 00:00:00.029 --> 00:00:03.630 Judith: Welcome to Berry's In the Interim podcast, where we explore the 00:00:03.630 --> 00:00:07.420 cutting edge of innovative clinical trial design for the pharmaceutical and 00:00:07.420 --> 00:00:09.640 medical industries, and so much more. 00:00:10.020 --> 00:00:10.699 Let's dive in. 00:00:12.011 --> 00:00:12.371 Scott: All right. 00:00:12.411 --> 00:00:15.681 Welcome everybody back to In The Interim. 00:00:16.711 --> 00:00:18.051 your host, Scott Berry. 00:00:18.321 --> 00:00:20.601 Uh, I'm your co-host, Scott Berry. 00:00:21.101 --> 00:00:27.511 Uh, my, my co-host, uh, my common co-host, Kurt Veily, is back. 00:00:27.531 --> 00:00:29.421 Though Kurt, it's been a bit of a, 00:00:29.886 --> 00:00:30.526 Kert Viele: Benny Gap. 00:00:30.966 --> 00:00:32.195 Yep, happy to be here. 00:00:32.266 --> 00:00:32.986 Happy to be here 00:00:33.431 --> 00:00:38.321 Scott: we, we may find out, uh, if Kurt has any pet peeves for the today. 00:00:38.441 --> 00:00:44.711 But get to the topic of the day, and you know, we are somewhere in 00:00:44.711 --> 00:00:51.571 the 60-plus episodes, Kurt, and we have not done an episode on 00:00:51.571 --> 00:00:53.871 response adaptive randomization. 00:00:54.771 --> 00:00:56.591 Something that is… 00:00:56.911 --> 00:00:59.371 makes people passionate. 00:00:59.631 --> 00:01:02.211 Uh, it's controversial. 00:01:02.451 --> 00:01:07.681 Um, it's, it's a great topic for innovative trial designs. 00:01:07.771 --> 00:01:13.581 I think it's especially a great topic as the world coming, the world 00:01:13.591 --> 00:01:15.641 of clinical trials is evolving. 00:01:15.641 --> 00:01:20.251 You know, what is, what is the role of, of response adaptive randomization in it? 00:01:20.261 --> 00:01:21.661 What are our thoughts on it? 00:01:22.081 --> 00:01:25.081 Um, lots of critics of response adaptive randomization. 00:01:25.901 --> 00:01:30.211 So let's, let's get to the, to the bottom, to the top, to the sides 00:01:30.591 --> 00:01:32.841 of response adaptive randomization 00:01:33.758 --> 00:01:34.688 Kert Viele: Sounds great 00:01:36.015 --> 00:01:36.525 Scott: Okay. 00:01:36.965 --> 00:01:40.155 So it's a controversial topic. 00:01:40.185 --> 00:01:47.005 Let's, let's, let's, uh, start with what is response adaptive randomization? 00:01:47.015 --> 00:01:52.255 Remember, my, my, my, my wife loves to tune in on these, and 00:01:52.515 --> 00:01:55.915 she's gonna wanna make sure that everybody understands the topic. 00:01:55.915 --> 00:02:00.085 So explain to Tammy what response adaptive randomization is. 00:02:00.416 --> 00:02:00.566 Kert Viele: All right. 00:02:00.576 --> 00:02:01.706 First off, hi, Tammy. 00:02:01.746 --> 00:02:02.356 Good to see you. 00:02:02.776 --> 00:02:07.036 Um, the, uh, so the idea behind RAR, response adaptive randomization - we'll 00:02:07.036 --> 00:02:12.746 abbreviate it from here on out - um, is essentially, uh, all adaptive 00:02:12.746 --> 00:02:16.695 trials, they do interim analyses, and they make some kind of adjustment to 00:02:16.696 --> 00:02:18.916 the trial at those interim analyses. 00:02:19.366 --> 00:02:25.126 In RAR, the adjustment that you're making is to change the allocation probabilities. 00:02:25.406 --> 00:02:29.606 And usually, what we would do is we would increase allocation to 00:02:29.616 --> 00:02:31.406 arms that are performing better. 00:02:31.595 --> 00:02:35.206 We decrease allocation to arms that are performing worse. 00:02:35.266 --> 00:02:38.816 The basic idea is to do a couple things. 00:02:38.826 --> 00:02:42.576 One is to try to treat patients better in the trial, try to get them the 00:02:42.585 --> 00:02:47.376 better-performing arms, and we hope we get some better inferences at the same time. 00:02:47.406 --> 00:02:49.886 But I know we're gonna touch on that when we get into some of the 00:02:49.896 --> 00:02:53.366 controversies because that sometimes happens and sometimes doesn't 00:02:54.375 --> 00:02:54.465 Scott: Yeah. 00:02:55.175 --> 00:02:59.285 So e-examples of, of how this may be, and we, we've been involved 00:02:59.285 --> 00:03:03.615 in multiple trials where response adaptive randomization has been used. 00:03:04.135 --> 00:03:11.925 And so a couple of those, the, the BAN2401 trial was a trial for, of lecanemab, which 00:03:11.925 --> 00:03:14.335 is now approved for Alzheimer's disease. 00:03:15.005 --> 00:03:22.625 Their phase two trial was a placebo and five active arms of lecanemab. 00:03:22.945 --> 00:03:27.225 It was actually two frequencies and three doses. 00:03:27.595 --> 00:03:33.805 And those five arms, they, they did a hundred and ninety-six patients enrolled 00:03:33.805 --> 00:03:40.045 where it was initially fixed randomization to those, those six arms in the trial. 00:03:40.415 --> 00:03:46.095 An interim analysis took place, and a new randomization probability 00:03:46.095 --> 00:03:47.775 was set for each of the arms. 00:03:48.755 --> 00:03:52.445 in-- essentially, the control was a fixed rate. 00:03:53.145 --> 00:03:56.785 slightly different than that, but, but for all intents and purposes, the control 00:03:56.785 --> 00:04:00.195 was fixed, and the other five could vary. 00:04:00.205 --> 00:04:05.165 With the remaining probability, a new randomization vector was created. 00:04:05.535 --> 00:04:11.605 For the next period of time, patients were randomized u-using those probabilities, 00:04:11.995 --> 00:04:17.305 and then fifty patients later enrolled, a new interim was done, and this was 00:04:17.305 --> 00:04:19.405 repeated during the course of the trial. 00:04:20.125 --> 00:04:24.835 And it ended up that two of those doses, the two higher 00:04:24.865 --> 00:04:29.675 doses with the two frequencies, were the most patients explored. 00:04:29.675 --> 00:04:34.615 One of them was moved to phase three, and it, it was, uh, uh, 00:04:34.615 --> 00:04:36.635 successful in a phase three trial. 00:04:37.585 --> 00:04:42.135 goal of that phase two trial was to find the best dose. 00:04:42.205 --> 00:04:45.615 What is the best arm, and how good is that arm? 00:04:46.919 --> 00:04:51.299 And we'll come back to whether RAR is good or not, but that, that was 00:04:51.299 --> 00:04:56.859 an example of, of response-adaptive randomization being used. 00:04:57.799 --> 00:04:59.469 I'll point out one more. 00:04:59.479 --> 00:05:02.759 We'll, we'll probably talk about other trials and how they do it and 00:05:02.759 --> 00:05:03.999 the, the good and the bad of it. 00:05:04.449 --> 00:05:08.169 Uh, I'll touch on another one just because I've done two recent podcasts 00:05:08.179 --> 00:05:12.049 on it, and we are doing a podcast on the results of the ICECAP trial. 00:05:12.719 --> 00:05:16.959 It-- By the way, it's been recorded, and we're just waiting for JAMA to publish 00:05:16.989 --> 00:05:22.059 the article, and then we're gonna release the discussion of the ICECAP trial. 00:05:22.509 --> 00:05:24.959 The ICECAP trial had 10 arms. 00:05:26.019 --> 00:05:31.309 arms were the duration of hypothermic cooling for the treatment of cardiac 00:05:31.309 --> 00:05:34.119 arrest, post-cardiac arrest resuscitation. 00:05:34.409 --> 00:05:39.119 You go to the, uh, the emergency room, and they do different durations of 00:05:39.119 --> 00:05:45.059 hypothermic cooling, and the goal was to find the best duration and largely 00:05:45.189 --> 00:05:47.009 was there an increasing dose response? 00:05:47.009 --> 00:05:48.529 Largely, what is the best duration? 00:05:48.609 --> 00:05:52.499 Was the goal of that, and there were 10 arms in that trial 00:05:54.066 --> 00:05:58.016 Kert Viele: And I-- you should also add that the, the lecanemab story, 00:05:58.376 --> 00:06:02.736 you published that article as well, so people can go back and read that and 00:06:02.736 --> 00:06:04.746 see what happened interim to interim. 00:06:04.746 --> 00:06:05.555 I thought that was… 00:06:05.626 --> 00:06:08.076 A lot of times we don't publish that kind of information. 00:06:08.076 --> 00:06:09.476 I thought it was good that it's out there 00:06:09.927 --> 00:06:16.337 Scott: Yeah, it's the only trial I know of that published the actual interim reports 00:06:16.357 --> 00:06:21.127 that were done at each of, I think there were 18 or 19 interim analyses, and it, 00:06:21.147 --> 00:06:27.047 it shows the quantities that were used to calculate the randomization probabilities. 00:06:27.047 --> 00:06:29.477 It shows what each one were at all the interims. 00:06:29.777 --> 00:06:30.957 You can see those reports. 00:06:30.957 --> 00:06:31.807 Fantastic. 00:06:31.987 --> 00:06:34.727 I'm trying to get the ICECAP people to publish that, and we will. 00:06:35.007 --> 00:06:38.717 We'll, we'll show each of the interims for that as well so people can follow along. 00:06:38.907 --> 00:06:42.507 Watch the movie at home, um, uh, uh, of the trial. 00:06:42.597 --> 00:06:46.247 So that gives a little bit of a flavor in just explaining 00:06:46.487 --> 00:06:48.277 response-adaptive randomization. 00:06:49.727 --> 00:06:52.097 Now, there, there… 00:06:52.247 --> 00:06:57.057 Uh, it-- to some extent, it sounds fantastic, um, uh, as this. 00:06:57.357 --> 00:07:01.477 If I'm a patient, I feel like I would like to be in a trial 00:07:01.477 --> 00:07:05.587 that uses response-adaptive randomization just at face value. 00:07:05.937 --> 00:07:11.147 Um, they-- At, at, at the same time within this, I think there are a 00:07:11.147 --> 00:07:16.257 lot of people who are unfamiliar with clinical trials, that when I 00:07:16.267 --> 00:07:22.027 explain to them adaptive trials, they look at me I have three heads. 00:07:22.027 --> 00:07:25.637 Like, don't all trials do this? 00:07:25.697 --> 00:07:31.537 Shouldn't the r- the allocation of treatments the latter part of the 00:07:31.537 --> 00:07:35.597 trial be informed by the earlier part of the trial and what's given to them? 00:07:35.597 --> 00:07:40.607 And when you describe that, no, most of them not, um, they're surprised by that. 00:07:41.487 --> 00:07:42.897 So there's, there's that part. 00:07:42.897 --> 00:07:45.467 But there are, there are critics of it. 00:07:45.527 --> 00:07:46.267 There are… 00:07:46.347 --> 00:07:50.387 A-and, a-and I, I don't know where we fall in this, Kurt. 00:07:50.437 --> 00:07:52.827 Um, I don't necessarily… 00:07:52.867 --> 00:07:58.967 I'm not pro-RAR, uh, as I, I don't think there's value in that. 00:07:58.967 --> 00:08:06.447 It's a tool in the trial to g- in some cases, get better answers, some cases 00:08:06.447 --> 00:08:11.737 treat patients better, but it's a tool available to us as trial designers, 00:08:11.907 --> 00:08:14.227 and it has good and bad in it. 00:08:14.637 --> 00:08:20.347 I think there-- it's fair to say there are, there are authors out there who 00:08:20.377 --> 00:08:25.337 are, against RAR and write articles. 00:08:25.677 --> 00:08:28.397 "Don't, don't fall for the RAR trick. 00:08:28.427 --> 00:08:29.147 Don't do it. 00:08:29.147 --> 00:08:29.917 It's bad." 00:08:30.317 --> 00:08:33.477 Uh, but it is quite controversial in the literature 00:08:34.470 --> 00:08:37.039 Kert Viele: At least we're very persuasive because we, we're lay- 00:08:37.079 --> 00:08:39.880 laying out traps and we're getting people to do our, our stuff. 00:08:39.890 --> 00:08:42.939 But ironically, we don't-- how many of these do we actually do? 00:08:42.939 --> 00:08:44.030 We certainly do them. 00:08:44.270 --> 00:08:48.360 I mean, we do, you know, hundreds of trials over the years, but it's 00:08:48.360 --> 00:08:51.610 certainly not 60% of these involve RAR. 00:08:51.840 --> 00:08:56.180 It's particular cases where we've done it and particular cases where we haven't 00:08:58.319 --> 00:08:59.119 Scott: Very much so. 00:08:59.309 --> 00:09:02.419 Um, let, so let's talk about what, what those cases. 00:09:02.419 --> 00:09:10.479 So i- if, if, um, um, what, what are the, what are the critics of RAR? 00:09:10.479 --> 00:09:15.179 What, where does this perhaps not go well? 00:09:15.888 --> 00:09:18.248 Kert Viele: So, and well, this gets at whether, you know, we're critics 00:09:18.248 --> 00:09:20.528 of RAR for certain cases as well. 00:09:20.538 --> 00:09:24.218 So, I mean, the, the central thing, I think what you said from the patient 00:09:24.218 --> 00:09:27.338 standpoint, you know, obviously if you're enrolling late in the 00:09:27.347 --> 00:09:30.568 trial, you wanna be, "Hey, a lot of people have generated information. 00:09:30.568 --> 00:09:31.988 I wanna get the benefit of that." 00:09:31.988 --> 00:09:34.097 I would fully understand that perspective. 00:09:34.797 --> 00:09:39.838 There's a scientific aspect that the purpose of the trial is to generate 00:09:39.858 --> 00:09:41.848 information beyond the trial. 00:09:41.848 --> 00:09:45.168 If you're doing, in lecanemab, there are millions of people who 00:09:45.168 --> 00:09:49.668 are suffering from Alzheimer's that want a good and effective drug. 00:09:49.767 --> 00:09:54.298 So we wanna keep them in mind while we're also trying to help patients in the trial. 00:09:55.878 --> 00:10:00.688 One thing that happens is to get a lo-- I'm not gonna do a lot of math here at 00:10:00.688 --> 00:10:05.878 all, of course, but if you're looking at estimating a treatment effect, so the 00:10:05.878 --> 00:10:07.998 sample sizes in each of the arms matter. 00:10:07.998 --> 00:10:09.748 You're comparing control to treatment. 00:10:09.977 --> 00:10:14.198 If you can make both of those arms have bigger sample sizes, 00:10:14.408 --> 00:10:16.198 you're gonna get better estimates. 00:10:16.208 --> 00:10:17.768 You're gonna get more information. 00:10:18.358 --> 00:10:22.208 So now the question is, well, how does RAR actually do that? 00:10:22.528 --> 00:10:25.957 Well, RAR does that by increasing allocation to some 00:10:25.957 --> 00:10:27.857 arms, decreasing it to others. 00:10:28.438 --> 00:10:32.518 If you're in a two-arm trial, for example, this is the classic and a 00:10:32.518 --> 00:10:35.808 lot of the papers that are against RAR, they focus on two arms with 00:10:36.008 --> 00:10:38.067 reason to say it's problematic here. 00:10:38.738 --> 00:10:43.458 You increase the treatment allocation potentially, but you decrease the control. 00:10:44.078 --> 00:10:47.798 And this gets into kind of a robbing Peter to pay Paul situation. 00:10:48.428 --> 00:10:50.338 You can actually decrease… 00:10:50.368 --> 00:10:56.478 You, you estimate treatment effects worse in two-arm RAR s- than you 00:10:56.478 --> 00:10:58.718 would in doing a fixed trial. 00:10:58.828 --> 00:11:00.508 So a lot of people have noted this. 00:11:00.508 --> 00:11:01.468 There's a problem. 00:11:01.687 --> 00:11:04.538 Two-arm trials, we don't do RAR very often. 00:11:04.578 --> 00:11:07.468 In this case, I think it's a really specialized issue. 00:11:07.928 --> 00:11:12.608 All that's gonna be turned on its head in a multiple-arm setting, uh, and I 00:11:12.608 --> 00:11:13.828 think we'll get to that in a minute. 00:11:13.828 --> 00:11:18.718 But two arms is a very-- If you read the literature, two arms is a very, very, 00:11:18.738 --> 00:11:23.228 very controversial case, and I think we agree with the, the limitations there 00:11:24.327 --> 00:11:28.897 Scott: So, so a lot of critics of RAR will point out that in a 00:11:28.897 --> 00:11:35.767 two-arm trial, uh, moving away from one-to-one, reduces your power. 00:11:35.937 --> 00:11:41.107 It reduces your, uh, uh, i- inferences, your inferences about 00:11:41.107 --> 00:11:42.687 the, the difference between them. 00:11:42.937 --> 00:11:48.007 Now, there are some odd cases where, uh, it might be good to reduce one, 00:11:48.007 --> 00:11:51.287 the variability's different in the two arms and, and, and all of that. 00:11:51.287 --> 00:11:53.707 But largely, you reduce power in that. 00:11:53.927 --> 00:11:57.847 And then they make a sweeping argument that so you should never do RAR 00:11:58.558 --> 00:11:59.838 Kert Viele: Yeah, so that's my pet peeve. 00:11:59.888 --> 00:12:00.918 You wanted a pet peeve. 00:12:00.938 --> 00:12:06.398 This is-- My, my pet peeve is how overgeneralized the RAR literature is. 00:12:06.398 --> 00:12:09.028 Uh, uh, RAR, it's a lot of different methods. 00:12:09.028 --> 00:12:10.538 You can do it a lot of different ways. 00:12:10.538 --> 00:12:12.438 You can do it in a lot of different cases. 00:12:12.538 --> 00:12:17.288 A lot of papers are written, "I'm gonna explore this version of RAR 00:12:17.338 --> 00:12:21.658 in this setting, but I'm gonna generalize it to everything," and 00:12:21.668 --> 00:12:23.058 that just doesn't serve us well 00:12:23.313 --> 00:12:26.103 Scott: And they say, "Look, this method doesn't work here. 00:12:26.423 --> 00:12:28.023 RAR doesn't work," and they 00:12:28.182 --> 00:12:29.793 generalize it to every possible 00:12:29.992 --> 00:12:31.403 RAR cases of that. 00:12:31.608 --> 00:12:31.688 Kert Viele: So 00:12:32.073 --> 00:12:36.613 Scott: Okay, so in-- So you talked about the two-arm case, the two, the 00:12:36.613 --> 00:12:41.683 two-arm setting, and so that would be a specialized scenario where the 00:12:41.713 --> 00:12:44.953 goal isn't entirely treatment effect. 00:12:45.213 --> 00:12:47.963 There, there may be other goals, and we, and we'll come to that a little bit. 00:12:49.703 --> 00:12:56.583 So let's just, uh, take, uh, another criticism of it is temporal trends. 00:12:57.203 --> 00:13:01.793 So what is the, the, the, the criticism about temporal trends? 00:13:02.686 --> 00:13:05.596 Kert Viele: So there's something really magical about doing fixed 00:13:05.596 --> 00:13:07.686 randomization, not doing RAR. 00:13:08.056 --> 00:13:14.516 So if, if I run a trial and I enroll patients one-to-one, and I have patients 00:13:14.516 --> 00:13:18.385 that enroll early in the trial, I have patients that enroll late in the trial. 00:13:19.096 --> 00:13:24.266 If I look at the treatment and control groups, however those split between 00:13:24.266 --> 00:13:28.366 how many patients came in early and how many came in late, they're the same 00:13:28.396 --> 00:13:31.256 between treatment and control because I've done one-to-one the whole way. 00:13:31.256 --> 00:13:34.186 I've equalized allocation the whole way through the trial. 00:13:34.806 --> 00:13:37.645 If I do RAR, that doesn't happen necessarily. 00:13:37.936 --> 00:13:43.146 If I start the trial one-to-one and then shift to three-to-one, for example, or 00:13:43.156 --> 00:13:47.155 some people would say nine-to-one, uh, which I don't think we get to very often. 00:13:47.535 --> 00:13:53.706 Uh, but in any case, if you do that, so what's gonna happen is you go three-to-one 00:13:53.706 --> 00:13:56.006 in favor of treatment later in the trial. 00:13:56.586 --> 00:14:00.386 You look at the treatment subjects, they're more late patients. 00:14:00.876 --> 00:14:05.136 You look at the control subjects, there are fewer late patients. 00:14:05.596 --> 00:14:10.716 If there's any kind of anything going on, and the classic example is COVID, where 00:14:10.716 --> 00:14:15.036 there's a new variant coming in and it-- people act-- react to it differently, 00:14:15.516 --> 00:14:21.276 that is gonna generate a bias, and that is potentially really, really bad. 00:14:21.356 --> 00:14:24.416 So I mean, it depends on how big the bias is, but it can completely 00:14:24.416 --> 00:14:26.016 invalidate your conclusions. 00:14:27.096 --> 00:14:29.716 Typically, we would add terms to our model. 00:14:29.716 --> 00:14:34.836 We'd model time to address this, uh, but that's generally what the issue is, is you 00:14:34.836 --> 00:14:39.096 do need to take active steps to correct for that and make sure you're robust. 00:14:40.249 --> 00:14:43.439 Scott: Oh, that, that creates biases. 00:14:43.719 --> 00:14:48.409 And that term, it creates biases, always depends on how you're analyzing the data. 00:14:48.688 --> 00:14:48.978 Kert Viele: Right 00:14:49.229 --> 00:14:53.859 Scott: you ignore the fact that it's different over time and it's not part 00:14:53.859 --> 00:14:56.619 of the model, then it could cause bias. 00:14:57.089 --> 00:15:03.859 If you're adjusting by time, then, know, assuming you have additive 00:15:03.879 --> 00:15:08.109 effects across time, then it's not biased within that setting. 00:15:08.109 --> 00:15:12.469 So there are, there are ways to adjust for time if it's a concern. 00:15:12.819 --> 00:15:16.929 There, there are some cases where we have much less concern about time effects. 00:15:16.929 --> 00:15:20.559 In some cases, we have larger effects about, uh, larger concerns about time. 00:15:20.799 --> 00:15:22.679 So there are ways to adjust for that. 00:15:23.869 --> 00:15:31.539 Okay, um, there's one other potential criticism, which is the rare circumstance 00:15:31.549 --> 00:15:37.879 where people may be un- uh, unblinded to the, what's the allocation is, is 00:15:37.879 --> 00:15:42.229 that you may learn one treatment is doing better than another one just 00:15:42.229 --> 00:15:43.989 by the fact it's being given more. 00:15:44.809 --> 00:15:49.289 in a blinded trial where patients are blinded, investigators are 00:15:49.289 --> 00:15:53.979 blinded, nobody would know that a particular treatment is given more. 00:15:54.169 --> 00:15:58.055 In trials where they're unblinded, this becomes also a potential 00:15:58.055 --> 00:15:58.799 criticism, um, uh, uh, of, of RAR 00:16:01.406 --> 00:16:04.806 Kert Viele: Yeah, and among many other criticisms of an unblinded trial. 00:16:04.866 --> 00:16:09.076 Lots of things can happen in those settings, and RAR may make them worse, so 00:16:09.871 --> 00:16:13.031 Scott: Okay, so we've set up potential negatives. 00:16:13.031 --> 00:16:15.451 This unblinding aspect of it is a potential negative. 00:16:16.461 --> 00:16:22.031 trends is a potential negative to it, and in a two-arm trial, you reduce power. 00:16:22.201 --> 00:16:24.001 So, uh, o- okay, 00:16:24.132 --> 00:16:25.412 Kert Viele: sounds, sounds awful, Scott 00:16:25.591 --> 00:16:26.451 Scott: sounds, sounds bad. 00:16:26.711 --> 00:16:27.871 So what are the potential 00:16:28.030 --> 00:16:29.170 Kert Viele: So, so 00:16:29.621 --> 00:16:30.381 Scott: of, of, of RAR? 00:16:30.501 --> 00:16:32.721 We, we started, it seemed like a positive thing. 00:16:32.734 --> 00:16:33.034 Kert Viele: So, 00:16:33.061 --> 00:16:34.051 Scott: positives of RAR? 00:16:34.724 --> 00:16:37.994 Kert Viele: well, so there is a case where RAR is a good match. 00:16:38.214 --> 00:16:42.624 So in that case, the one we often use it for is situations where 00:16:42.624 --> 00:16:44.874 we are looking at multiple arms. 00:16:44.874 --> 00:16:48.764 We're not in a two, two-arm setting, and we're looking for the best 00:16:48.773 --> 00:16:50.653 arm among those alternatives. 00:16:50.664 --> 00:16:53.674 So the classic examples here would be dose-finding. 00:16:54.054 --> 00:16:57.574 I'm not particularly worried about, you know, which arm is the fourth-best. 00:16:57.574 --> 00:16:59.044 I wanna know what is the best. 00:16:59.514 --> 00:17:03.904 Um, in the epidemic settings, the pandemic settings, we wanna know what's 00:17:03.904 --> 00:17:05.924 the best treatment for, for patients. 00:17:05.924 --> 00:17:08.143 We don't necessarily need to know the others. 00:17:08.884 --> 00:17:15.273 In those cases, what RAR does for you is because you are increasing allocation 00:17:15.273 --> 00:17:19.073 to the best arm, it's the one doing the best, it's the one you're gonna raise, 00:17:19.464 --> 00:17:21.934 you get more information on that best arm. 00:17:22.384 --> 00:17:23.523 You wanna be careful here. 00:17:23.553 --> 00:17:26.154 This is the thing we've always been telling people is you wanna make sure 00:17:26.154 --> 00:17:30.614 you maintain the control allocation because that's part of your comparison. 00:17:30.614 --> 00:17:33.644 If you're gonna do an estimate at the end to compare to control, you need 00:17:33.644 --> 00:17:36.184 controls, so don't, don't skimp on those. 00:17:36.764 --> 00:17:41.364 But in those cases, RAR will give you more power 'cause it gets 00:17:41.364 --> 00:17:45.343 more patients on the, on the arms that matter in your comparison. 00:17:45.823 --> 00:17:51.714 It gets you lower bias, lower, um, uh, lower variance, so 00:17:51.724 --> 00:17:53.603 basically you get better estimates. 00:17:53.764 --> 00:17:56.664 All of these inferential things start working in your favor 00:17:58.317 --> 00:18:04.297 Scott: So i-in a, in a setting, and looked at in a, in a treat-- potential 00:18:04.347 --> 00:18:09.537 trial for Ebola, for example, you might have four active treatments that could 00:18:09.547 --> 00:18:11.397 be brought in as a treatment for it. 00:18:11.877 --> 00:18:16.237 If you're increasing the probability of one or two of those treatments and 00:18:16.237 --> 00:18:20.347 keeping control the same, you have a better idea of figuring out the best. 00:18:20.367 --> 00:18:26.007 But now you've reduced the, the randomization to arm four, in a setting, 00:18:26.017 --> 00:18:28.047 the question is, do you care about that? 00:18:28.377 --> 00:18:30.397 So power is such an odd… 00:18:30.497 --> 00:18:32.787 Uh, it's not an odd thing, but it's a really important thing to 00:18:32.787 --> 00:18:35.677 unders- make sure we understand what are we trying to do in the trial. 00:18:36.027 --> 00:18:37.507 What are the goals of the trial? 00:18:37.677 --> 00:18:41.427 If it's trying to identify every possible treatment that's good or 00:18:41.427 --> 00:18:45.807 bad, you might not wanna do RIR 'cause it's not finding the best. 00:18:46.117 --> 00:18:50.767 If it's about finding the best, then identifying the effect, it can increase 00:18:50.767 --> 00:18:56.427 the power, and dose finding is a great example of that because you don't care, 00:18:56.737 --> 00:18:59.837 uh, about a-arms that aren't as effective. 00:18:59.837 --> 00:19:01.547 You're trying to identify the best. 00:19:01.567 --> 00:19:07.087 And so it's a perfect example, and it's commonly where we do it the most at Barry 00:19:07.097 --> 00:19:12.367 is a dose finding trial, uh, where, where it can create a better trial design. 00:19:13.607 --> 00:19:14.067 Okay. 00:19:14.107 --> 00:19:20.987 Now, at the same time, so if somebody enters into a trial and, um, there's 00:19:20.987 --> 00:19:28.567 this framework that's set up in these trials of a VIP, where suppose you 00:19:28.567 --> 00:19:34.117 were running a trial and a VIP came in and you wanted to treat the patient 00:19:34.117 --> 00:19:37.557 as well, and you could look at the data and assign one of the arms. 00:19:37.957 --> 00:19:41.477 As the trial goes on, you're more and more likely to put that patient on a 00:19:41.477 --> 00:19:46.337 better arm because you're learning, and you would treat that VIP better. 00:19:46.827 --> 00:19:50.257 Now, we don't do that in trials, and VIPs don't get to come in 00:19:50.257 --> 00:19:51.577 and look at the data of it. 00:19:52.007 --> 00:19:57.457 Um, but that, i-if we're, if we're increasing the probability to arms 00:19:57.457 --> 00:20:01.547 that are doing better and we're learning more efficiently, we're 00:20:01.617 --> 00:20:03.637 treating patients better in the trial. 00:20:05.027 --> 00:20:08.847 And so a patient that comes in later in the trial is more likely to be 00:20:08.847 --> 00:20:12.787 given a more effective arm in the trial 00:20:13.757 --> 00:20:15.277 Kert Viele: So we've talked about this. 00:20:15.658 --> 00:20:19.527 Um, it probably doesn't happen in trials very often, but we've talked 00:20:19.527 --> 00:20:23.637 about this example in the context of a learning healthcare system where 00:20:23.697 --> 00:20:27.658 the, the whole healthcare system is trying to improve itself, but 00:20:27.668 --> 00:20:30.918 somebody walks in the door going, "I, I don't want you to randomize me. 00:20:30.918 --> 00:20:34.547 You just look at the data and tell me which one's the best, and I want that." 00:20:35.017 --> 00:20:38.648 And you can imagine situations where that's-- I mean, we talk about the 00:20:38.658 --> 00:20:40.507 ethics of it for a long period of time. 00:20:40.967 --> 00:20:42.937 Uh, but it's at least conceivable. 00:20:43.327 --> 00:20:49.747 The nice thing about RAR is in that ethical case, if you were doing 00:20:49.747 --> 00:20:55.037 fixed randomization, that VIP coming in wanting special treatment, they 00:20:55.037 --> 00:20:58.617 actually get treated much better than the average patient because the 00:20:58.617 --> 00:21:01.747 average patient's getting randomized, not making use of information. 00:21:02.447 --> 00:21:07.367 If you're doing RAR, what's happening is the difference between that 00:21:07.467 --> 00:21:12.967 VI-VIP and the average patient in the trial is actually pretty darn small. 00:21:12.977 --> 00:21:16.227 It's a couple-- You know, if you're doing mortality, it's a couple percent 00:21:16.237 --> 00:21:18.388 rather than twenty, twenty-five percent. 00:21:18.697 --> 00:21:24.178 So it really helps on getting treat-- patients treated well and 00:21:24.237 --> 00:21:31.127 can-- the, the idea that we're still learning as well as, um, as well 00:21:31.127 --> 00:21:32.587 as making use of that information 00:21:33.447 --> 00:21:33.657 Scott: Yep. 00:21:34.127 --> 00:21:34.367 Yep. 00:21:35.457 --> 00:21:40.867 so from a perspective, and I started this off with if I'm, if I'm a 00:21:40.867 --> 00:21:45.997 patient going into a trial and there are two trials, and one of them is 00:21:45.997 --> 00:21:50.037 doing RAR and one of them is not, I'd rather go in the trial with RAR. 00:21:50.077 --> 00:21:53.937 I find that a, a, a better place to go. 00:21:54.877 --> 00:22:00.177 interestingly, we do a number of trials, especially rare disease, we're treating 00:22:00.177 --> 00:22:06.307 a pediatric disease, um, a Duchenne muscular dystrophy, where you might do 00:22:06.307 --> 00:22:08.237 two to one or three to one randomization. 00:22:08.597 --> 00:22:13.327 There's a s- very strong preference of a patient to enter into a trial 00:22:13.517 --> 00:22:17.677 where they're more likely to be put on the active arm, and it's, it's 00:22:17.677 --> 00:22:23.277 done as a tool to increase enrollment wi- within the trial to, to, to 00:22:23.377 --> 00:22:27.187 increase the likelihood patients would agree to enter into that trial 00:22:27.507 --> 00:22:30.237 because it's favoring the active arm. 00:22:30.687 --> 00:22:35.777 Now you've got a procedure that learns from the data and, and goes to different 00:22:35.797 --> 00:22:41.097 arms within it, which is, which is a perception generally, I think, that 00:22:41.097 --> 00:22:43.307 patients would rather be in that trial 00:22:44.030 --> 00:22:47.970 Kert Viele: And I, I think we ought to add kind of to the, the objections to RAR 00:22:47.970 --> 00:22:53.580 here that, you know, we simulate these trials and how are we doing this RAR? 00:22:53.589 --> 00:22:55.840 How are we increasing the allocation? 00:22:56.230 --> 00:22:58.030 You can, of course, do this badly. 00:22:58.310 --> 00:23:01.449 Uh, when we present this to people who have never, you know, done 00:23:01.450 --> 00:23:05.570 adaptive trials before, they-- the natural question is, well, what if 00:23:05.570 --> 00:23:10.310 you overreact to early data and the early data is just noisy and wrong? 00:23:10.730 --> 00:23:16.009 And certainly, the answer is if you do RAR badly, you can overreact to that. 00:23:16.030 --> 00:23:19.020 There are papers that have gone through, you know, doing interims 00:23:19.020 --> 00:23:23.149 at a sample size of three where, hey, this isn't working out well. 00:23:23.160 --> 00:23:24.590 So you wanna simulate this. 00:23:24.590 --> 00:23:29.110 You wanna make sure that you do this in a way where you don't overreact, but 00:23:29.129 --> 00:23:31.050 you're making use of the information. 00:23:31.330 --> 00:23:35.679 And that requires, uh, basically a seasoned practitioner in 00:23:35.680 --> 00:23:37.069 order to find the right balance 00:23:37.735 --> 00:23:38.085 Scott: Yeah. 00:23:38.085 --> 00:23:43.315 So, uh, we would never enter into a trial, put an algorithm together 00:23:43.315 --> 00:23:47.295 that does RAR, just hope it works. 00:23:47.925 --> 00:23:51.255 Uh, we simulate thousands of trials. 00:23:51.525 --> 00:23:51.975 We… 00:23:52.155 --> 00:23:53.125 Millions of trials. 00:23:53.125 --> 00:23:57.295 We make sure under a huge range of scenario, it's getting better answers, 00:23:57.325 --> 00:24:02.335 or it's accomplishing the goals that it's intended to improve upon. 00:24:02.795 --> 00:24:07.105 Uh, if it's that trial itself, we wanna improve the outcome of patients, 00:24:07.105 --> 00:24:08.605 if it's to learn the right dose. 00:24:08.985 --> 00:24:12.495 Uh, so we simulate them e-extensively ahead of time. 00:24:12.755 --> 00:24:19.435 There's a famous example of an RAR algorithm where, in hindsight, I'm not 00:24:19.435 --> 00:24:24.315 sure that they like the outcome of it, um, within it, and it's the ECMO trial. 00:24:24.765 --> 00:24:31.905 Uh, uh, Bartlett, years ago, ran a trial, and it was, um, newborns that 00:24:31.905 --> 00:24:35.945 had particular respiratory issues. 00:24:36.295 --> 00:24:42.095 They were being randomized between conventional care and ECMO, 00:24:43.005 --> 00:24:45.665 an ECMO machine, um, for them. 00:24:46.145 --> 00:24:49.035 Now, the, the-- What was the design? 00:24:49.035 --> 00:24:54.245 It was called a play the winner design and, or Polya urn design, 00:24:54.545 --> 00:25:00.205 where initially there's a bowl and there's two balls in the bowl. 00:25:00.215 --> 00:25:04.285 One is red for conventional, one is blue for ECMO. 00:25:05.385 --> 00:25:11.775 And the first patient comes in, you grab a ball, and whatever ball is selected, that, 00:25:11.815 --> 00:25:19.471 that's assigned to the patient If that patient is a success and they survive, 00:25:19.551 --> 00:25:25.581 survival was the endpoint, you put back the ball on the therapy they were given. 00:25:25.581 --> 00:25:29.171 So if they were given the blue therapy and they lived, you put 00:25:29.171 --> 00:25:31.251 another blue ball in the, in the bowl. 00:25:31.991 --> 00:25:35.201 And then the next patient that comes in has a two-thirds chance 00:25:35.201 --> 00:25:37.061 of be given the, the blue therapy. 00:25:37.541 --> 00:25:43.251 If they're given blue and they don't survive, you put a red ball in 00:25:43.251 --> 00:25:46.871 the, in the urn, and now it's more likely the patient would get red. 00:25:47.221 --> 00:25:51.751 And every patient that, that continues and you put more and more balls in the 00:25:51.771 --> 00:25:57.061 urn i- in there, and therapy's doing better have more balls in there, therapy's 00:25:57.101 --> 00:25:59.461 doing worse, the other one has more. 00:25:59.731 --> 00:26:05.351 And it was a, it was a procedure for doing response adaptive randomization that. 00:26:06.091 --> 00:26:10.551 Now, what happened in the trial is the first patient that came 00:26:10.551 --> 00:26:15.261 in was randomized to conventional care, and the patient died. 00:26:15.921 --> 00:26:19.771 Um, and so a new ball was put in for ECMO. 00:26:20.261 --> 00:26:23.921 The next patient that came in got ECMO and survived. 00:26:24.711 --> 00:26:28.341 now a new ball is put in, so now it's 75% ECMO. 00:26:29.521 --> 00:26:35.381 it went on a run of like 27 consecutive ECMO patients, 00:26:35.771 --> 00:26:37.631 and they were all surviving. 00:26:37.761 --> 00:26:39.611 I think there may have been a death in there. 00:26:39.911 --> 00:26:45.721 But the trial ended up, I believe, uh, like 26 out of 27 on ECMO 00:26:46.191 --> 00:26:48.891 and oh for one or oh for two. 00:26:48.901 --> 00:26:53.231 I think at the end they even added a few more conventional therapies on there. 00:26:53.531 --> 00:26:58.471 But it ended up incredibly disparate, uh, uh, in the trial. 00:26:58.801 --> 00:27:01.941 Now, it turns out ECMO is highly effective. 00:27:01.951 --> 00:27:03.241 It's better than conventional. 00:27:03.241 --> 00:27:04.521 We know that now. 00:27:04.841 --> 00:27:09.491 But the trial was greatly criticized, and I had the, I had the benefit once 00:27:09.491 --> 00:27:16.021 of asking Bartlett, "Had you seen a simulation that showed what happened 00:27:16.021 --> 00:27:20.161 in the trial, would you have said that was good, I'm glad it did it?" 00:27:20.171 --> 00:27:20.901 He said, "No. 00:27:21.021 --> 00:27:21.341 No. 00:27:21.521 --> 00:27:23.061 We would have done something different." 00:27:23.341 --> 00:27:27.851 And that's the value of simulation, to be able to evaluate that particular algorithm 00:27:29.182 --> 00:27:32.752 Kert Viele: So I, uh, one aspect of that, so all of these play the winner, 00:27:32.752 --> 00:27:37.012 the poly-polyarms schemes, they come out of a particular literature, 00:27:37.012 --> 00:27:41.502 which obviously, you know, your father Don's heavily contributed to. 00:27:41.972 --> 00:27:44.131 But w- there is one aspect of that. 00:27:44.172 --> 00:27:49.882 A lot of that literature is based on trying to treat a sequence 00:27:49.882 --> 00:27:54.992 of patients well without having any regulatory aspect to it. 00:27:55.002 --> 00:27:58.612 So at no point do you have to generate, you know, P less than .025, 00:27:58.612 --> 00:28:00.052 so to speak. 00:28:00.781 --> 00:28:06.411 Um, one thing that people should keep in mind is that when you put that kind 00:28:06.462 --> 00:28:10.712 of regulatory requirement on it, you need a different kind of evidence. 00:28:10.752 --> 00:28:14.422 And so that's one of the things more modern RAR methods do is they 00:28:14.422 --> 00:28:18.621 take into account the regulatory environment as well, which is a 00:28:18.632 --> 00:28:20.302 change over the past 40 years. 00:28:21.159 --> 00:28:22.519 Scott: Wh- whi- which is doable. 00:28:22.686 --> 00:28:23.045 Kert Viele: Yeah 00:28:23.269 --> 00:28:26.939 Scott: there are guidance documents that say response adaptive randomization 00:28:27.039 --> 00:28:29.249 is, can be done in confirmatory trials. 00:28:29.599 --> 00:28:33.369 Um, it's probably not very common because phase three a 00:28:33.369 --> 00:28:35.179 lot of times are two-arm trials 00:28:35.358 --> 00:28:35.678 Kert Viele: Yeah 00:28:35.729 --> 00:28:38.789 Scott: and, and there may not be benefit in a t- two-arm trial, 00:28:38.789 --> 00:28:40.489 especially in that regulatory setting. 00:28:40.489 --> 00:28:45.019 It's unlikely that you would do RAR in a two-arm phase three trial. 00:28:45.275 --> 00:28:48.275 Kert Viele: Well, you, you have examples of seamless two, three trials that have 00:28:48.275 --> 00:28:52.605 done RAR in the first part and then lowered, lowered the number of doses 00:28:52.951 --> 00:28:53.181 Scott: Right. 00:28:53.191 --> 00:28:58.161 So Eli Lilly's treatment Trulicity, dulaglutide, uh, it's one of the 00:28:58.191 --> 00:29:04.151 earliest GLP-1 inhibitors, was seven-arm phase two trial that at 00:29:04.151 --> 00:29:06.411 some point could trigger phase three. 00:29:07.181 --> 00:29:13.241 it selected two doses to move forward into a fixed randomization phase three portion. 00:29:13.551 --> 00:29:18.161 By the way, it included data from the first part, uh, in the final analysis 00:29:18.161 --> 00:29:21.181 of that, which was done through response adaptive randomization. 00:29:21.591 --> 00:29:23.901 And it honed in on the 1.5 00:29:23.901 --> 00:29:24.941 milligram dose. 00:29:25.171 --> 00:29:32.731 Higher doses were having, safety issues, high heart rate, high blood pressure. 00:29:32.931 --> 00:29:35.951 It moved away from them, and 1.5 00:29:35.961 --> 00:29:39.461 was the dose that moved forward, eventually has done incredibly 00:29:39.461 --> 00:29:42.551 well and, you know, billions of dollars of year treatment. 00:29:42.551 --> 00:29:45.531 So that, that is certainly done in seamless two, three trials, where the 00:29:45.531 --> 00:29:47.481 first part is to find the right dose. 00:29:47.951 --> 00:29:54.591 An interesting side effect of that is that the original design in that trial 00:29:54.591 --> 00:29:57.501 was three arms in the phase two part. 00:29:58.191 --> 00:30:03.241 And when they looked at expanding that to a range of seven doses, 00:30:03.591 --> 00:30:08.311 that increases the sample size by that factor, you know, seven thirds 00:30:08.311 --> 00:30:10.061 and it's, oh, that, that's too big. 00:30:10.351 --> 00:30:14.191 But by doing response adaptive randomization and honing in on a 00:30:14.191 --> 00:30:19.061 particular part of the dose response curve, y- the sample size isn't 00:30:19.111 --> 00:30:23.741 seven thirds, and the sample size wasn't even any bigger to do seven 00:30:23.741 --> 00:30:27.161 doses than three doses because of the modeling in the trial. 00:30:27.541 --> 00:30:29.621 So there's a huge benefit to that. 00:30:29.621 --> 00:30:31.921 The ICECAP is very similar to that. 00:30:32.201 --> 00:30:37.281 ICECAP was three durations, but they wanted a wider range of durations. 00:30:37.291 --> 00:30:41.611 They went to 10 with largely the same sample size because it hones 00:30:41.621 --> 00:30:46.271 in very quickly in the region of interest, uh, in the scenario. 00:30:46.271 --> 00:30:46.821 So there are… 00:30:47.251 --> 00:30:50.491 The, the higher power isn't just higher power. 00:30:50.491 --> 00:30:55.781 It may enable more doses, uh, to be used in a better trial design 00:30:56.446 --> 00:30:58.646 Kert Viele: And one thing that you talked about, well, you'd obviously 00:30:58.646 --> 00:31:01.755 talked about the TRILOGY trial for a long time, but the, uh, the 00:31:01.755 --> 00:31:03.586 notion that it incorporated safety. 00:31:03.586 --> 00:31:09.245 You, you were doing RAR not just on an efficacy measure, but a combination 00:31:09.555 --> 00:31:14.086 which allowed them to pick a dose that balanced several features. 00:31:14.086 --> 00:31:16.386 I forget what the features are, what were in that trial? 00:31:16.623 --> 00:31:21.313 Scott: clinical utility index that was HbA1c chains, 00:31:21.661 --> 00:31:22.021 Kert Viele: Yeah 00:31:23.553 --> 00:31:27.023 Scott: Uh, uh, amazingly in the world we're in, weight loss, which 00:31:27.023 --> 00:31:32.983 has become, the GLP-1s now approved solely for weight loss, um, and 00:31:32.983 --> 00:31:34.353 blood pressure and heart rate. 00:31:34.383 --> 00:31:38.173 So there are four endpoints that went into selecting the optimal 00:31:38.183 --> 00:31:41.553 therapeutic, uh, uh, dose, uh, in it. 00:31:43.043 --> 00:31:43.693 okay. 00:31:44.053 --> 00:31:47.663 Um, uh, within this, by the way, we should mention, uh, it's sort 00:31:47.663 --> 00:31:50.613 of a really interesting story when it w- in and of itself. 00:31:50.613 --> 00:31:57.903 You mentioned, uh, Don's work in this and bandit problems, uh, and its relationship 00:31:57.903 --> 00:31:59.693 to response adaptive randomization. 00:31:59.703 --> 00:32:03.963 But even going farther, back farther than this, largely, I think the first 00:32:03.963 --> 00:32:10.253 randomized trial, randomized clinical trial of humans, um, was in the 1940s. 00:32:10.253 --> 00:32:11.283 Uh, streptomycin, 00:32:11.464 --> 00:32:13.214 Kert Viele: 1948, streptomycin 00:32:14.703 --> 00:32:18.443 Scott: uh, treatment of that, fixed randomization, uh, two-arm 00:32:18.443 --> 00:32:19.853 trial, fixed randomization. 00:32:20.123 --> 00:32:25.993 But you can go back to 1933, and there's a paper by Thompson where he introduces 00:32:25.993 --> 00:32:32.203 response adaptive randomization, uh, to that, and it was a long-forgotten paper, 00:32:32.633 --> 00:32:38.653 um, within it that has now gr- been, been, uh, uh, cited many, many times, 00:32:38.653 --> 00:32:44.223 and it's even a little bit of the, uh, search, uh, uh, uh, uh, Google search 00:32:44.223 --> 00:32:48.513 things and, and bandit problems that has brought that paper back to life. 00:32:49.293 --> 00:32:50.704 Kert Viele: And not just brought it back to life. 00:32:50.704 --> 00:32:52.704 I mean, it's one of the more cited papers. 00:32:52.704 --> 00:32:54.744 I mean, talk about having to wait for fame. 00:32:55.094 --> 00:33:00.264 You publish it in 1933, you get very little attention for 70 years, 00:33:00.264 --> 00:33:04.513 and now suddenly you have 5,000 citations or whatever it's at now 00:33:04.583 --> 00:33:04.823 Scott: Yep. 00:33:05.703 --> 00:33:12.573 bandit problems is largely, it-- you've got multiple arms to pull, and you decide 00:33:12.573 --> 00:33:14.983 on which arm to allocate to a patient. 00:33:15.223 --> 00:33:18.073 Usually, they're done in a deterministic way. 00:33:18.383 --> 00:33:21.573 Oh, give them arm three, give them arm one. 00:33:21.813 --> 00:33:26.723 And usually, you've set up a goal of that particular trial, which can 00:33:26.723 --> 00:33:29.883 include a very large horizon that at some point you have to pick a 00:33:29.883 --> 00:33:31.913 treatment and go with one treatment. 00:33:32.323 --> 00:33:36.643 Um, but you set up a goal to, to save as many particular patients 00:33:36.643 --> 00:33:37.883 or have a particular outcome. 00:33:38.163 --> 00:33:43.523 And, and this was my father's dissertation work, was bandit problems, uh, within it. 00:33:43.523 --> 00:33:47.523 So closely related to response adaptive randomization is this whole 00:33:47.523 --> 00:33:52.663 literature of bandit problems, which have also been cited many, many times. 00:33:53.003 --> 00:33:59.303 Um, and many of this is now advertising, uh, online advertising. 00:33:59.473 --> 00:34:03.793 If Google has multiple ways in which it can present an ad to you, 00:34:04.053 --> 00:34:06.153 and I'm using Google just as a, a 00:34:06.270 --> 00:34:06.600 Kert Viele: Yeah 00:34:06.953 --> 00:34:10.743 Scott: of things, it can, uh, throw out three and, and find 00:34:10.743 --> 00:34:12.413 out how many clicks does it get? 00:34:12.453 --> 00:34:14.603 Well, maybe we'll try two, maybe we'll try one. 00:34:14.883 --> 00:34:19.443 Maybe we'll personalize it to individuals within that so they 00:34:19.443 --> 00:34:25.133 can be using this technology to increase the number of clicks 00:34:28.623 --> 00:34:29.203 Okay. 00:34:29.553 --> 00:34:36.323 Um, uh, now within the, the properties of this, I wanted-- platform 00:34:36.323 --> 00:34:40.173 trials have all, uh, brought out a really interesting part to this. 00:34:40.563 --> 00:34:46.703 So platform trials are those where we have multiple agents in the trial and 00:34:46.723 --> 00:34:51.053 a, a, a relatively new, uh, uh, thing. 00:34:51.443 --> 00:34:56.843 And we might have four e-experimental treatments and a 00:34:56.843 --> 00:34:59.943 control in the trial at one time. 00:35:00.913 --> 00:35:05.373 So this opens up the interesting question of do we wanna do response 00:35:05.373 --> 00:35:10.853 adaptive randomization platform trials, which largely are a multi-arm trial 00:35:13.726 --> 00:35:18.046 Kert Viele: So I, I think platform trials to me are we, we still haven't explored 00:35:18.046 --> 00:35:20.156 this in as much detail as we would like. 00:35:20.156 --> 00:35:24.556 We've done simulations for our own trials and so on, but RAR, it's a 00:35:24.556 --> 00:35:28.366 really different beast in platform trials, or at least it can be. 00:35:28.886 --> 00:35:33.015 If, if I'm doing an umbrella trial where I've got four arms, I've got to enroll 00:35:33.015 --> 00:35:39.256 two hundred patients, when I do RAR, if I'm increasing allocation to certain 00:35:39.276 --> 00:35:41.726 arms, I'm decreasing allocation to others. 00:35:42.456 --> 00:35:45.905 In a platform trial that's perpetual, so I'm gonna do four arms, I'm 00:35:45.905 --> 00:35:50.605 gonna replace an arm when I drop it, increasing the allocation doesn't 00:35:50.656 --> 00:35:52.556 necessarily lower sample size. 00:35:52.586 --> 00:35:54.366 It slows things down. 00:35:54.906 --> 00:35:59.526 And so you could say, "I'm gonna speed up the arms that look the most promising," 00:35:59.776 --> 00:36:01.765 but I'm not gonna abandon everything else. 00:36:01.766 --> 00:36:05.596 It's still there for me to get to it later, potentially. 00:36:05.886 --> 00:36:09.665 And I've been really interested in how all that plays out in practice, 00:36:09.825 --> 00:36:11.205 and we've done this both ways. 00:36:11.396 --> 00:36:15.206 But I, I think this is one of the open research areas is how this works. 00:36:15.847 --> 00:36:18.837 Scott: Yeah, one of the controversial things or the w- things that may 00:36:18.837 --> 00:36:22.917 be problematic is if there's four sponsors that own-- four separate 00:36:22.942 --> 00:36:23.282 Kert Viele: Yeah 00:36:23.497 --> 00:36:28.607 Scott: that own the drugs, and we accelerate sponsor A, it slows down 00:36:28.637 --> 00:36:33.177 sponsor B by, by, by definition, and that may be an undesirable 00:36:33.187 --> 00:36:36.307 thing to recruiting arms in a trial. 00:36:36.557 --> 00:36:41.107 So many of them in that scenario, phase two setting, don't use response 00:36:41.107 --> 00:36:42.657 adaptive randomization because 00:36:42.848 --> 00:36:45.898 Kert Viele: it would even be worse if we said we weren't going to explore an 00:36:45.908 --> 00:36:48.228 arm at all, much less than slow it down. 00:36:48.238 --> 00:36:51.268 That often generates more controversy with sponsors with reason, "Hey, 00:36:51.268 --> 00:36:52.728 we want you to give us an answer." 00:36:53.233 --> 00:36:53.443 Scott: Right. 00:36:53.503 --> 00:36:58.573 One trial where this was done with multi-sponsors, and it was generally 00:36:58.573 --> 00:37:04.533 perceived by all to be a good thing, is the I-SPY2 So the I-SPY2 trial is a Phase 00:37:04.563 --> 00:37:10.003 2 trial in neoadjuvant breast cancer, and there would be 20% fixed on the 00:37:10.003 --> 00:37:16.073 control, and then the remaining 80% was set up across the experimental arms that 00:37:16.073 --> 00:37:20.123 were in the trial, and it would impr- it would increase the, the probability. 00:37:20.623 --> 00:37:25.933 The, the really interesting thing there is breast cancer, we, we understand 00:37:25.933 --> 00:37:29.483 heterogeneity of disease in breast cancer better than most diseases. 00:37:31.123 --> 00:37:34.453 HER2 breast cancer is, HER2 positive is a different breast 00:37:34.453 --> 00:37:35.803 cancer than HER2 negative. 00:37:36.213 --> 00:37:42.013 So the trial was enrolling, uh, hormone receptor by HER2 status, 00:37:42.033 --> 00:37:43.443 also had MammaPrint status. 00:37:43.453 --> 00:37:48.933 So there's eight types of cancer that women are being enrolled within that. 00:37:48.933 --> 00:37:54.923 So if you came in with HER2 negative, hormone receptor negative, triple 00:37:54.923 --> 00:37:59.373 negative disease, you are have a different randomization probabilities 00:37:59.373 --> 00:38:03.113 over the four arms that are in the trial than somebody who's HER2 00:38:04.643 --> 00:38:07.203 hormone receptor negative within that. 00:38:07.503 --> 00:38:07.833 The… 00:38:08.683 --> 00:38:16.173 we might accelerate one drug for one type of cancer, but a different drug is 00:38:16.183 --> 00:38:21.823 being accelerated in another type, and they had a fixed 120-patient sample size. 00:38:22.193 --> 00:38:26.283 So they got more patients within the subtypes where they were 00:38:26.283 --> 00:38:30.583 doing better, and they might have gotten less in other settings. 00:38:30.593 --> 00:38:36.513 So you were treating patients better, but the, the sponsors perceived this to be a 00:38:36.513 --> 00:38:43.173 positive because of the multiple subgroups where RRR was done different by subgroup. 00:38:43.193 --> 00:38:45.823 So it wasn't just slowing down one for the other. 00:38:46.013 --> 00:38:48.943 One slows up one place and speeds up the other place. 00:38:49.143 --> 00:38:51.333 That was thought to be a very positive thing 00:38:51.656 --> 00:38:56.196 Kert Viele: Well, and it, it speeds up their phase three because if you know 00:38:56.256 --> 00:39:01.086 where your treatment has the biggest benefit, you can design a smaller 00:39:01.086 --> 00:39:03.185 trial to detect that in phase three. 00:39:03.345 --> 00:39:07.385 You enroll the people who benefit, and you don't dilute the effect through people who 00:39:07.385 --> 00:39:11.316 don't, and obviously the people who don't benefit can go to other trials where they 00:39:11.316 --> 00:39:13.895 may achieve benefit and benefit them there 00:39:15.207 --> 00:39:19.887 Scott: So, well, one of the places where I've run into the most controversy is, 00:39:19.887 --> 00:39:27.017 uh, so I-I've been involved in, uh, multiple dozens of trials that use RAR. 00:39:27.367 --> 00:39:28.717 Very happy with all of them. 00:39:28.717 --> 00:39:31.987 And really very happy with e-every use of RAR. 00:39:32.487 --> 00:39:37.107 We did have a couple issues in the REMAP-CAP trial, uh, so 00:39:37.107 --> 00:39:38.757 fair to sort of point that out. 00:39:38.807 --> 00:39:40.417 Uh, these have been made public. 00:39:40.757 --> 00:39:43.647 Where REMAP-CAP is multifactorial. 00:39:44.057 --> 00:39:49.697 So a patient that comes in could be randomized to drug A, yes or no. 00:39:50.837 --> 00:39:55.317 Dr- And then a different domain of treatments that you could get drug 00:39:55.327 --> 00:39:58.217 one, two, or three, uh, within that. 00:39:58.407 --> 00:40:02.217 And, you know, w- I think the most a patient's been randomized 00:40:02.217 --> 00:40:05.787 is seven different domains, simultaneously randomized that. 00:40:06.817 --> 00:40:09.477 it uses response adaptive randomization. 00:40:09.487 --> 00:40:13.837 Some of those domains are two, um, arm domains. 00:40:13.867 --> 00:40:16.817 There's a control, and then there's, yes, you get therapy. 00:40:17.127 --> 00:40:20.297 Should you get high-dose vitamin C, yes or no? 00:40:20.587 --> 00:40:22.877 That was one of the domains in the trial. 00:40:23.897 --> 00:40:31.767 Now, so it was deemed this is a trial that was treating a pandemic, COVID-19, 00:40:31.767 --> 00:40:33.507 and I'll, I'll be specific to that. 00:40:33.507 --> 00:40:37.067 We're also enrolling non-pandemic community-acquired 00:40:37.067 --> 00:40:38.677 pneumonia, uh, within that. 00:40:38.677 --> 00:40:43.357 But within the pandemic, it was thought that this was highly beneficial to 00:40:43.357 --> 00:40:45.257 do response adaptive randomization. 00:40:46.047 --> 00:40:51.747 was more likely that groups, uh, that, that, there would be a positive to 00:40:51.757 --> 00:40:56.107 entering into this trial if it was using the most up-to-date information. 00:40:56.117 --> 00:40:59.397 It's treating the pandemic while it's learning at the same time. 00:41:00.437 --> 00:41:07.527 Now, we did end up in a two-domain case of simvastatin, where it ended 00:41:07.527 --> 00:41:12.007 up about ninety percent probability for simvastatin versus no simvastatin. 00:41:12.717 --> 00:41:18.227 And it hovered around determining efficacy, and it probably took longer 00:41:19.257 --> 00:41:24.827 it to declare efficacy, uh, than it would have if it stayed one-to-one. 00:41:25.257 --> 00:41:28.837 But meanwhile, it was increasing the probability of patients getting 00:41:28.837 --> 00:41:30.987 simvastatin during the pandemic. 00:41:31.817 --> 00:41:36.987 So this was a case that was debated a lot as to whether RAR was a good thing 00:41:36.987 --> 00:41:38.987 or not in that particular scenario 00:41:39.870 --> 00:41:42.690 Kert Viele: So we've got-- There's another example in addition to the pandemic. 00:41:42.690 --> 00:41:47.880 We have a trial called PROSPECT, which ha- we've-- there's a protocol paper out. 00:41:47.880 --> 00:41:49.589 The results paper is not out yet. 00:41:49.589 --> 00:41:51.690 We're hoping that will come relatively soon. 00:41:52.219 --> 00:41:56.049 Um, but anyway, it is a two-by-two factorial experiment. 00:41:56.060 --> 00:42:00.099 So technically there are four arms, but you could think of it as doing 00:42:00.099 --> 00:42:03.850 two arms twice on the rows and columns of that two-by-two table. 00:42:04.039 --> 00:42:04.299 Scott: Yep 00:42:04.529 --> 00:42:08.900 Kert Viele: there were similar discussions there over this is-- PROSPECT is on, 00:42:09.240 --> 00:42:11.600 uh, respiratory distress in children. 00:42:11.969 --> 00:42:15.310 And so the notion was, hey, we want to treat these kids well 00:42:15.319 --> 00:42:16.890 while we're trying to learn. 00:42:17.130 --> 00:42:19.160 And that was exactly part of the debate there. 00:42:20.839 --> 00:42:24.969 Scott: Uh, one other case just to be upfront is we did have a domain where 00:42:24.969 --> 00:42:29.559 initially data were reported and flipped. 00:42:30.939 --> 00:42:37.169 Um, and so what happens is the RAR, intending to improve the better, um, 00:42:37.169 --> 00:42:39.989 went the wrong way, uh, within that. 00:42:40.769 --> 00:42:42.939 Lot, a, a number of lessons learned. 00:42:42.949 --> 00:42:46.479 Now, the final data that went into it was all fixed and all of that. 00:42:46.479 --> 00:42:51.369 So by the way, there's a, a, a strong operational burden upon, 00:42:51.859 --> 00:42:54.049 uh, I, I don't wanna say strong. 00:42:54.059 --> 00:42:58.569 There's an operational burden upon doing RAR, making sure the data's 00:42:58.569 --> 00:43:02.689 right that goes into the RAR, the very- various pieces of this. 00:43:02.959 --> 00:43:06.639 It's, uh, an important part of the story is the logistical part of doing RAR. 00:43:06.949 --> 00:43:09.319 Some settings, it's reasonably straightforward. 00:43:09.319 --> 00:43:14.379 I-Spy 2, it, it ran weekly automated, um, but you gotta get the data right 00:43:15.856 --> 00:43:19.206 Kert Viele: I remember doing the RACE trial and I would get 00:43:19.226 --> 00:43:24.076 data every 12 patients, and this was back in the old days. 00:43:24.076 --> 00:43:26.846 And so I would be sitting there, I basically put together the 00:43:26.846 --> 00:43:31.936 randomization table and it got replaced in the randomization system. 00:43:31.945 --> 00:43:36.476 But I was more or less manually doing that, which was not what 00:43:36.486 --> 00:43:38.506 you'd want if you can avoid it. 00:43:38.526 --> 00:43:42.696 I think everything worked out fine in this, but it's been good seeing more 00:43:42.696 --> 00:43:47.456 and more groups learn how to do this and be able to do this automatically. 00:43:47.726 --> 00:43:51.946 But as you said, there are still some snafus that occur and you 00:43:51.946 --> 00:43:55.516 want an experienced group doing this, not just on the design side, 00:43:55.516 --> 00:43:56.886 but on the implementation side 00:43:58.119 --> 00:43:58.369 Scott: Yep. 00:43:58.959 --> 00:44:02.899 Uh, I do wanna come back to something you said that I think in sort of looking 00:44:02.909 --> 00:44:05.869 forward to the future here, uh, of… 00:44:06.029 --> 00:44:10.439 I, I think platform trials, multi-arm experiments, uh, are 00:44:10.439 --> 00:44:13.919 growing in popularity, even in comparative effectiveness. 00:44:14.289 --> 00:44:17.689 I do think somewhat of the future is a learning healthcare system, 00:44:18.129 --> 00:44:21.659 where if you imagine where we are today, we do these experiments to 00:44:21.659 --> 00:44:25.309 learn the right therapy, but 99.9% 00:44:25.309 --> 00:44:29.098 of people are treated outside of those, and we don't learn from them at all. 00:44:29.899 --> 00:44:36.259 As we start to merge learning about different treatments with treating 00:44:36.259 --> 00:44:40.479 patients at the same time, we, we do experiments where the Belmont 00:44:40.479 --> 00:44:45.189 Report lays out that it's okay that these people are being experimented 00:44:45.189 --> 00:44:46.539 on, that we do one-to-one. 00:44:47.339 --> 00:44:50.579 But in a learning healthcare system where we want to treat patients 00:44:50.609 --> 00:44:55.089 better and learn about therapies, response-adaptive randomization 00:44:55.089 --> 00:44:57.239 is an incredibly powerful tool. 00:44:57.569 --> 00:45:02.439 If I owned a healthcare system and there was a particular treatment 00:45:02.439 --> 00:45:08.999 and there's five available drugs to treat that, uh, IBS, psychiatry, 00:45:09.719 --> 00:45:15.334 number of scenarios, radomizing patients to the different treatments, 00:45:16.034 --> 00:45:21.249 response-adaptive randomization, learning the right therapies, we don't 00:45:21.249 --> 00:45:25.629 learn from these patients at all i-in a randomized causal way for sure, 00:45:26.079 --> 00:45:28.369 uh, is an incredibly powerful thing. 00:45:28.369 --> 00:45:32.399 So I think response-adaptive randomization, while it's a powerful 00:45:32.399 --> 00:45:36.179 tool in the right setting, it's a bad tool in other settings, and 00:45:36.179 --> 00:45:37.429 that's, that's the part of it. 00:45:37.479 --> 00:45:45.179 Um, is it has a really strong future as I think healthcare becomes more 00:45:45.179 --> 00:45:47.279 integrated, learning healthcare. 00:45:47.539 --> 00:45:50.989 Uh, I think it, it, it's, it's a powerful future 00:45:52.118 --> 00:45:55.267 Kert Viele: And you're gonna be exploring combinations of therapies. 00:45:55.508 --> 00:45:59.957 All of this is gonna go together in ways that typically aren't done 00:45:59.997 --> 00:46:01.988 in, say, a hundred-patient trial. 00:46:02.198 --> 00:46:03.348 So just 'cause I can't 00:46:03.943 --> 00:46:08.483 Scott: And like the I-SPY 2 trial, we're gonna have a much better idea of the, how 00:46:08.593 --> 00:46:14.093 diseases are cl- similar but different, heterogeneity of treatment effect. 00:46:14.403 --> 00:46:19.883 Uh, all of this comes together where I think res- response adaptive randomization 00:46:19.893 --> 00:46:21.953 is, is going to be even more valuable. 00:46:22.243 --> 00:46:26.043 Thompson is going to, post 100 years, get more and more 00:46:26.043 --> 00:46:28.513 citations as, as time goes on. 00:46:29.193 --> 00:46:32.783 Uh, which I think is Thompson sampling, it's even called. 00:46:33.113 --> 00:46:33.793 Um, 00:46:33.822 --> 00:46:34.572 Kert Viele: Certain types? 00:46:34.622 --> 00:46:35.002 Yep 00:46:36.953 --> 00:46:37.723 Scott: All right. 00:46:37.723 --> 00:46:43.423 Well, it took us, it took us 60-plus episodes, uh, before we got to 00:46:43.423 --> 00:46:45.363 response adaptive randomization. 00:46:45.663 --> 00:46:51.133 I, I suspect it's not our last discussion of this, but, uh, one of 00:46:51.143 --> 00:46:55.323 my favorite topics and, and one of them that get- keeps people listening 00:46:57.703 --> 00:46:59.463 So thanks for joining, Kurt. 00:46:59.543 --> 00:47:02.943 Appreciate everybody out there tuning in today. 00:47:03.263 --> 00:47:06.313 Until next time, we'll be here in the interim 00:47:07.149 --> 00:47:07.600 Kert Viele: Thanks, Scott