WEBVTT NOTE This file was generated by Descript 00:00:00.029 --> 00:00:03.630 Judith: Welcome to Berry's In the Interim podcast, where we explore the 00:00:03.630 --> 00:00:07.639 cutting edge of innovative clinical trial design for the pharmaceutical and 00:00:07.639 --> 00:00:09.800 medical industries, and so much more. 00:00:10.340 --> 00:00:11.060 Let's dive in. 00:00:13.152 --> 00:00:16.502 Well, welcome everybody back to In The Interim. 00:00:16.742 --> 00:00:18.272 I'm your host, Scott Berry. 00:00:19.062 --> 00:00:23.142 And as you can tell, my studio has changed. 00:00:23.652 --> 00:00:29.232 So the new studio, which, um, uh, if you go back a bit seasonally, 00:00:29.792 --> 00:00:35.182 that, um, I am here in northern Minnesota, and you can see beautiful. 00:00:35.182 --> 00:00:38.912 That's actually, uh, that, that's not a, that's not a 00:00:38.912 --> 00:00:42.132 virtual, uh, background to that. 00:00:42.132 --> 00:00:48.412 That is, that is, uh, the woods outside of our little cabin here in Minnesota. 00:00:48.412 --> 00:00:53.192 We get to escape the heat of Austin, Texas and spend some time 00:00:53.192 --> 00:00:56.392 here in the woods of Minnesota. 00:00:56.672 --> 00:01:01.372 So I'm in a little, uh, uh, office, uh, above the garage 00:01:01.372 --> 00:01:05.362 here in northern Minnesota, and I appreciate you joining me. 00:01:06.732 --> 00:01:16.082 So I, I wanna introduce today's topic with a bit of a panicked email and 00:01:16.082 --> 00:01:21.402 then phone call I got in October 2016. 00:01:23.272 --> 00:01:31.072 The, the panic was from my father, uh, my collaborator on I-SPY 2. 00:01:32.162 --> 00:01:36.632 And you can go back to previous episodes of In The Interim 00:01:36.632 --> 00:01:38.762 looking at the I-SPY 2 trial. 00:01:38.762 --> 00:01:43.142 We actually did, uh, two consecutive episodes about the trial. 00:01:43.142 --> 00:01:46.182 Fantastic, fantastic trial. 00:01:46.642 --> 00:01:51.152 I'll, I'll, I'll introduce a little bit, but it is… 00:01:51.432 --> 00:01:56.882 A, a important part of the story is the primary endpoint in the trial. 00:01:56.882 --> 00:02:02.042 The trial is I-SPY 2 neoadjuvant breast cancer. 00:02:03.132 --> 00:02:08.362 And the important part of the neoadjuvant part of th-this is that what that means is 00:02:08.852 --> 00:02:13.362 rather than, uh, traditionally surgery… 00:02:13.392 --> 00:02:16.392 You're diagnosed with breast cancer. 00:02:17.252 --> 00:02:22.132 A surgeon goes in and removes as much as the breast-- of the breast cancer 00:02:22.762 --> 00:02:30.298 as they can, and then you're given therapies That's adjuvant treatment. 00:02:30.688 --> 00:02:38.518 Neoadjuvant is where you don't go in and remove the tumor immediately. 00:02:39.138 --> 00:02:45.818 You give treatment, and six months later, surgery is done. 00:02:46.568 --> 00:02:53.718 And the-- a-an advantage of that is you get to see whether or not the treatment 00:02:53.718 --> 00:02:56.588 had any benefit on the tumor itself. 00:02:57.728 --> 00:03:06.498 The primary endpoint in I-SPY 2 was-- it, it was pathologic complete response. 00:03:07.318 --> 00:03:14.428 That means that when surgery is done six months after the start of the trial with-- 00:03:14.428 --> 00:03:17.848 for a patient, is that the cancer's gone. 00:03:19.238 --> 00:03:21.468 There's-- It's a complete response. 00:03:21.568 --> 00:03:25.428 There's no evidence of tumor at the time of surgery. 00:03:25.768 --> 00:03:29.498 And again, this is the statistician telling you the endpoint. 00:03:29.928 --> 00:03:34.388 A, a, um, an oncologist, breast cancer oncologist could do a much better 00:03:34.388 --> 00:03:39.858 job of this, but I'll give you the statistical needed parts of this. 00:03:40.278 --> 00:03:45.038 So that surgery is done six months after, and there's no evidence of cancer. 00:03:45.038 --> 00:03:46.848 The, the, the tumor's gone. 00:03:47.548 --> 00:03:51.318 Meaning that the treatment that you started and initiated 00:03:51.318 --> 00:03:55.898 six months earlier had really strong benefit for that patient. 00:03:57.038 --> 00:04:02.418 A, a, a lack of pathologic complete response would be that surgery is 00:04:02.418 --> 00:04:08.108 done, and there's tumor remaining that the, the surgeon removes at that time. 00:04:09.588 --> 00:04:10.158 Now 00:04:12.318 --> 00:04:15.528 That endpoint is a, is a six-month endpoint. 00:04:16.638 --> 00:04:24.292 In the trial, we don't know the result of that endpoint for a patient for six months 00:04:27.072 --> 00:04:36.242 Now, in the trial, there are earlier readings within a patient that might 00:04:36.242 --> 00:04:38.982 be predictive of that response. 00:04:39.882 --> 00:04:45.462 There's an MRI that's taken one month, three months, and 00:04:45.462 --> 00:04:48.452 then actually before surgery. 00:04:48.792 --> 00:04:53.692 It's, it's, we call it six months, but it's relatively, um, um, 00:04:54.342 --> 00:05:00.942 proximal to when surgery is done, and that MRI gives an estimate of 00:05:01.072 --> 00:05:03.612 the amount of tumor that is there. 00:05:05.202 --> 00:05:11.542 And now, now we, uh, we'll go back and even at six months, for example, 00:05:11.542 --> 00:05:17.022 the MRI might say there's no tumor left, but surgery reveals some lay or 00:05:17.072 --> 00:05:19.532 some, some, some aspects of, of tumor. 00:05:19.842 --> 00:05:22.122 So even at six months it's not perfect. 00:05:23.812 --> 00:05:31.212 Now, in the trial, that-- those values, the MRI values, are used 00:05:31.492 --> 00:05:33.412 within a patient in the modeling. 00:05:34.332 --> 00:05:35.502 I'll come back to that. 00:05:35.502 --> 00:05:40.932 What, what is the, what is the, the, the email and phone call panic that 00:05:40.932 --> 00:05:48.102 I got is that pembrolizumab, which one-- it was one of the treatments in 00:05:48.102 --> 00:05:51.742 the trial, had triggered graduation. 00:05:52.752 --> 00:06:01.762 Graduation is a, a, is an adaptive outcome in the trial where there's at least an 00:06:01.762 --> 00:06:09.102 eighty-five percent probability that that treatment would beat control in a phase 00:06:09.102 --> 00:06:14.522 III trial of pathologic complete response. 00:06:14.812 --> 00:06:19.192 Now, it's uncertain the role of pathologic complete response 00:06:19.192 --> 00:06:26.142 and do we need disease-free, uh, survival, uh, uh, as an endpoint? 00:06:26.142 --> 00:06:28.052 Do we need a different endpoint in this? 00:06:28.052 --> 00:06:32.092 And what's the role of pathologic complete re-response in breast cancer? 00:06:32.422 --> 00:06:33.862 Let, let's not get into that. 00:06:33.862 --> 00:06:38.402 But in the, in the trial, that's the-- that's one of the outcomes, 00:06:38.402 --> 00:06:43.522 and think of that as a positive, uh, uh, outcome that has graduated. 00:06:43.522 --> 00:06:46.912 It means enrollment in I-SPY stops. 00:06:47.592 --> 00:06:50.002 All patients would go through six months. 00:06:50.002 --> 00:06:54.312 They would continue on their therapy, but no new enrollment would be given. 00:06:54.312 --> 00:06:57.232 Essentially, the treatment has had a positive outcome. 00:06:57.542 --> 00:07:01.772 I-SPY 2 is a platform trial with many therapies going through it, 00:07:02.082 --> 00:07:04.192 pembrolizumab being one of those. 00:07:06.082 --> 00:07:14.162 Now, the panic was, and, and I'll back up even a little bit more, is that in 00:07:14.162 --> 00:07:21.452 I-SPY 2, Berry Consultants wrote the code that runs that adaptive trial. 00:07:22.322 --> 00:07:28.632 The trial uses response adaptive randomization across multiple subtypes 00:07:28.632 --> 00:07:35.972 of disease, hormone receptor status, HER2 status, and the, the adaptive design 00:07:35.972 --> 00:07:38.562 was driven by Bayesian modeling of that. 00:07:38.872 --> 00:07:43.542 And Berry Consultants and I was involved in writing the code for that. 00:07:45.362 --> 00:07:47.992 Now, coming back to that, what's the panic? 00:07:48.522 --> 00:07:56.422 At the time that it triggered for graduation, the estimated pathologic 00:07:56.422 --> 00:08:04.662 complete response rate for pembrolizumab was sixty percent, and the control rate 00:08:04.662 --> 00:08:06.802 had an estimate of twenty-three percent. 00:08:08.162 --> 00:08:12.182 Very, very successful in the trial. 00:08:13.182 --> 00:08:20.022 The panic was only one patient at that time had an observed outcome 00:08:20.022 --> 00:08:22.322 of pathologic complete response. 00:08:22.682 --> 00:08:28.352 Had had the six-month surgery and we knew the data on one patient out 00:08:28.352 --> 00:08:30.352 of the patients on pembrolizumab. 00:08:31.932 --> 00:08:35.182 And the panic was, what's happening? 00:08:35.362 --> 00:08:37.982 Is, is the modeling broken? 00:08:39.102 --> 00:08:43.462 We have one patient, and it estimates a sixty percent response 00:08:43.462 --> 00:08:47.202 rate compared to the twenty-three percent on control, control. 00:08:47.342 --> 00:08:49.052 Something must be broken 00:08:51.272 --> 00:08:54.032 Okay, so that's the panic. 00:08:55.022 --> 00:08:58.932 Now, I'll come back to, to that part of the story, but I wanna 00:08:58.932 --> 00:09:06.822 shift things to adaptive designs, and I-SPY 2 was an adaptive trial. 00:09:06.822 --> 00:09:10.952 I'll come back and talk a little bit more about I-SPY 2 and this October 00:09:10.952 --> 00:09:15.242 2016 panic, uh, uh, from I-SPY 2. 00:09:16.202 --> 00:09:18.872 But let's talk a little bit about adaptive trials. 00:09:19.622 --> 00:09:22.592 So what is an adaptive trial? 00:09:22.932 --> 00:09:29.892 An adaptive trial is a, a clinical trial that has pre-specified 00:09:30.382 --> 00:09:32.122 dynamic aspects to it. 00:09:32.612 --> 00:09:38.372 I try to not use the word change because the protocol is set up to 00:09:38.372 --> 00:09:41.752 have these dynamic pieces to it. 00:09:42.122 --> 00:09:49.612 So in I-SPY 2, there's adaptive allocation to the different treatment arms. 00:09:50.312 --> 00:09:54.172 There's triggering of graduation of an arm. 00:09:54.502 --> 00:09:56.482 All of that's pre-specified. 00:09:56.512 --> 00:09:58.372 The protocol doesn't change. 00:09:58.592 --> 00:10:01.322 It says in the protocol these things will happen. 00:10:02.802 --> 00:10:09.772 These are key aspects of the trial that are dynamic and by 00:10:09.772 --> 00:10:12.262 design can happen in the trial. 00:10:13.392 --> 00:10:16.812 Yes, from the perspective of the design, it feels like these are 00:10:16.812 --> 00:10:19.022 things that are changing in the trial. 00:10:19.642 --> 00:10:23.462 You know, we stop enrolling to pembrolizumab when it 00:10:23.462 --> 00:10:26.182 hits a trigger of graduation. 00:10:26.492 --> 00:10:31.132 The role of that is to accelerate development of it to get it to Phase 3. 00:10:31.512 --> 00:10:33.422 I-SPY 2 is a Phase 2 trial. 00:10:35.542 --> 00:10:43.332 Now, during the trial, these things are triggered by data in the trial itself 00:10:45.794 --> 00:10:49.694 Now that's it seems… 00:10:50.004 --> 00:10:55.184 When, when I explain to people one of the things I do as a statistician, 00:10:55.214 --> 00:10:59.764 the main thing Barry does is build adaptive trials, and I've been building 00:10:59.764 --> 00:11:07.324 adaptive trials for, for 26 plus years now, is it, it seems strange 00:11:07.404 --> 00:11:10.274 that, "Okay, I don't understand. 00:11:10.634 --> 00:11:12.974 You mean normally we don't… 00:11:13.514 --> 00:11:15.364 the trial doesn't change?" 00:11:16.264 --> 00:11:21.184 And to explain that is really the opposite of an adaptive design is a fixed trial. 00:11:21.954 --> 00:11:26.154 We- w- at the beginning of the trial in a fixed trial, you say, "Let's 00:11:26.154 --> 00:11:28.574 enroll 200 patients on a control. 00:11:28.834 --> 00:11:34.164 Let's enroll 200 patients on a treatment, and let's wait till all 00:11:34.164 --> 00:11:39.514 of those patients get six-month data, and then we'll look at the data and 00:11:39.514 --> 00:11:44.044 figure out whether pembrolizumab or whatever treatment it is works. 00:11:44.844 --> 00:11:47.584 And then we make the decisions going forward." 00:11:47.734 --> 00:11:49.354 That's a fixed trial. 00:11:50.744 --> 00:11:55.284 Now, many times in a fixed trial, you look at the data and you say, "Oh, shoot, 00:11:55.754 --> 00:11:57.654 I wish the sample size was smaller. 00:11:57.654 --> 00:12:00.984 We might've known this treatment was effective long before we 00:12:00.984 --> 00:12:03.354 got to, to, to 200 on each arm." 00:12:04.254 --> 00:12:08.084 Uh, in the, in I-SPY 2 it's 120 is the maximum, so you… 00:12:08.084 --> 00:12:11.424 long before you get to 120, you might think that treatment 00:12:11.424 --> 00:12:12.904 works or it doesn't work. 00:12:13.444 --> 00:12:17.274 Or I wish we'd have put different kind of patients on that treatment 00:12:17.274 --> 00:12:18.634 compared to another treatment. 00:12:19.104 --> 00:12:21.494 I wish we would've changed doses. 00:12:21.764 --> 00:12:26.144 But generally larger outside of I-SPY 2, we could be changing 00:12:26.654 --> 00:12:28.474 doses in a, in a trial. 00:12:28.704 --> 00:12:31.744 We could be altering the patients enrolled in a trial. 00:12:32.034 --> 00:12:36.164 We could move it to a different stage of trial where it selects a dose and 00:12:36.164 --> 00:12:40.844 it moves to a, a confirmatory part of the trial, and I'll talk about 00:12:40.844 --> 00:12:43.724 a trial that, that did that today. 00:12:44.174 --> 00:12:52.042 So lots of things in the trial can, can be dynamic Now what's, 00:12:52.212 --> 00:12:56.202 what, why is that, what's, what's the promise of adaptive trials? 00:12:56.432 --> 00:13:01.152 We think about fixed trials where you say, "Let's go enroll 200 patients on 00:13:01.152 --> 00:13:04.842 each arm, and we'll look at the data when all of the patients have reached 00:13:05.152 --> 00:13:07.642 a time point that we consider primary." 00:13:08.252 --> 00:13:12.162 We look at the data and we learn something from that, and maybe we approve a 00:13:12.162 --> 00:13:16.692 treatment, maybe we go to phase three, maybe we run fa- another phase two. 00:13:16.692 --> 00:13:21.192 Whatever it is, the role of the trial, that's a fixed trial. 00:13:21.682 --> 00:13:29.152 The promise of this is that we can change this trial in ways that improve the trial. 00:13:30.272 --> 00:13:36.042 By, by changing the allocation, by changing the doses, by 00:13:36.042 --> 00:13:39.882 accelerating to another stage, we make the trial design better. 00:13:40.192 --> 00:13:41.692 That's really promising. 00:13:42.062 --> 00:13:46.272 There are a lot of things in trials, in a fixed trial, we guess at. 00:13:46.762 --> 00:13:48.622 We guess at the effect size. 00:13:48.622 --> 00:13:52.572 We guess at which patients might, might benefit from it. 00:13:52.982 --> 00:13:55.162 We, we guess at the doses. 00:13:55.162 --> 00:13:57.702 We, we guess at a lot in the trial. 00:13:57.702 --> 00:14:01.712 We guess at the variability of the outcome, the, the 00:14:01.712 --> 00:14:04.482 prevalence of events in trials. 00:14:04.482 --> 00:14:05.852 We guess at a lot. 00:14:06.532 --> 00:14:10.092 The promise of adaptive trials is during the course of the trial, 00:14:10.092 --> 00:14:11.682 you learn a lot about that. 00:14:12.002 --> 00:14:18.092 If you look at the data in the trial itself, you learn how 00:14:18.092 --> 00:14:19.462 effective the treatment is. 00:14:19.462 --> 00:14:21.312 You learn about variability. 00:14:21.582 --> 00:14:24.932 You learn about the behavior of different treatments, the behavior 00:14:24.932 --> 00:14:29.892 of different doses, and then we make the trial better given that. 00:14:29.892 --> 00:14:35.002 Had we known that information before the trial started, we 00:14:35.002 --> 00:14:36.812 would've designed a better trial. 00:14:37.582 --> 00:14:42.632 That's the promise of adaptive trials, that we learn things to 00:14:42.632 --> 00:14:44.872 make that trial more efficient. 00:14:45.272 --> 00:14:46.872 We make the engine better. 00:14:47.782 --> 00:14:52.722 It's a, it's incredibly promising thing of, of adaptive trials. 00:14:53.382 --> 00:14:58.022 A- a- and if you think about it, the opposite of it is you close your eyes 00:14:58.602 --> 00:15:03.592 and you hope everything you designed four years later was the right thing to do. 00:15:05.232 --> 00:15:09.242 So that's the promise of an adaptive t- uh, of an adaptive trial. 00:15:10.062 --> 00:15:14.352 Now, the crux of it is as that adaptive trial is going, 00:15:14.742 --> 00:15:17.192 we need mechanisms to learn. 00:15:17.982 --> 00:15:23.482 There's data and, and data's ge- being generated in the trial. 00:15:24.182 --> 00:15:32.254 We need to learn from that data If you think about I-SPY 2, we need to learn from 00:15:32.254 --> 00:15:38.594 the pathologic complete response outcomes that come out of that trial at six months. 00:15:39.224 --> 00:15:44.414 In an Alzheimer's trial, we might be interested in the 18-month response. 00:15:45.504 --> 00:15:53.904 In a, in a, a trial of a vaccine, we're interested in, uh, events 00:15:53.904 --> 00:15:57.684 that are somebody comes up with the disease that the vaccine's 00:15:58.324 --> 00:16:03.244 supposed to combat, and we're waiting for events to happen in that. 00:16:03.514 --> 00:16:05.764 That becomes the thing we're learning from. 00:16:06.414 --> 00:16:14.764 So we need models in the trial itself as a learning mechanism, and those models 00:16:14.764 --> 00:16:17.714 drive the dynamic things in the trial. 00:16:18.094 --> 00:16:19.674 That's, that's what we need. 00:16:19.674 --> 00:16:23.454 So there's two key parts to an adaptive trial. 00:16:24.014 --> 00:16:29.364 We need to learn so that we make changes, and then when we design the 00:16:29.364 --> 00:16:33.254 trial, we figure out what changes in the trial make learning more 00:16:33.254 --> 00:16:37.624 efficient, make the trial better, treat patients better in the trial. 00:16:37.944 --> 00:16:41.814 All of those things are the dynamic changes, but we need 00:16:41.814 --> 00:16:43.164 the data in the trial to learn. 00:16:43.164 --> 00:16:47.384 If we don't-- If, if we try to make adaptations and we've learned nothing 00:16:47.384 --> 00:16:54.454 in the trial, you know, it, it, it's all based on, on, on things we thought going 00:16:54.454 --> 00:17:00.184 in, which is we already designed the trial, uh, a-as well as we can for that. 00:17:00.774 --> 00:17:03.184 So it's really these two key things. 00:17:03.184 --> 00:17:08.634 We need to learn from the data, and that drives changes in the trial that make it 00:17:08.634 --> 00:17:10.674 more efficient, make it a better trial. 00:17:11.744 --> 00:17:17.204 So in that I-SPY 2 example, graduation is an adaptation in the trial. 00:17:18.784 --> 00:17:24.914 We need to learn from the data to say, "Ah, this treatment should graduate." 00:17:26.594 --> 00:17:32.254 Now, in that circumstance of the panicked email was we have 00:17:32.254 --> 00:17:36.084 one patient at six months. 00:17:36.214 --> 00:17:37.994 The patient was a, a, a… 00:17:38.614 --> 00:17:40.664 was a, a positive response. 00:17:40.664 --> 00:17:44.714 It was a pathologic complete response, but that's one. 00:17:45.074 --> 00:17:50.944 How are-- How is your learning mechanism saying we have at least an eighty-five 00:17:50.944 --> 00:17:54.454 percent chance that it's better than a twenty-three percent response rate? 00:17:54.704 --> 00:17:58.964 If it's based on one out of one, that's-- we're not gonna come up with 00:17:58.964 --> 00:18:01.114 an eighty-five percent probability. 00:18:01.114 --> 00:18:01.824 The data's not strong enough. 00:18:03.324 --> 00:18:09.804 In that trial, we have built models to accelerate the learning. 00:18:11.784 --> 00:18:15.144 Now, you come back to this whole question about adaptive trial. 00:18:15.144 --> 00:18:20.644 If we're waiting for everybody to get to six months, the trial's slower 00:18:23.294 --> 00:18:31.364 The trial to make changes, to make dynamic adaptations, has to wait a long time, has 00:18:31.364 --> 00:18:33.634 to wait for patients to get to six months. 00:18:33.634 --> 00:18:36.634 Meanwhile, we've enrolled a bunch of patients earlier than that. 00:18:36.944 --> 00:18:39.534 Have we allocated them correctly? 00:18:40.524 --> 00:18:45.064 Now, in the trial, I, I, I brought up these MRIs that are done 00:18:45.064 --> 00:18:46.964 at one month and three months. 00:18:47.964 --> 00:18:54.654 In I-SPY 2, we built a model that looks at the correlation between the MRI, 00:18:54.654 --> 00:19:00.724 which is a quantitative measure of the reduction in tumor from baseline. 00:19:00.974 --> 00:19:05.274 Hundred percent reduction is getting to pathologic complete response 00:19:06.114 --> 00:19:07.534 at one month and three months. 00:19:07.534 --> 00:19:12.404 If there's no tumor in the MRI, we think that patient is highly likely 00:19:12.404 --> 00:19:14.624 to be a pathologic complete response. 00:19:15.334 --> 00:19:24.414 And hence, patients that have taken pembrolizumab that have MRI values that 00:19:24.414 --> 00:19:29.494 are really positive or really negative inform what we think the pathologic 00:19:29.494 --> 00:19:34.574 complete response is through what I'm gonna call a longitudinal model. 00:19:35.944 --> 00:19:40.814 So the longitudinal model is the statistical mechanism that's 00:19:40.814 --> 00:19:48.374 learning what does one-month MRI predict about six-month PCR? 00:19:49.094 --> 00:19:55.084 What does three months MRI predict about six-month PCR? 00:19:55.534 --> 00:20:01.394 What does six-month MRI predict about six-month pathologic 00:20:01.394 --> 00:20:02.864 comp-complete response? 00:20:02.864 --> 00:20:08.574 Surgery happens after that, not too far from it, so it's not a huge benefit 00:20:08.574 --> 00:20:12.034 to the trial at six months, but three months can speed up the learning. 00:20:12.064 --> 00:20:13.944 One month can speed up the learning. 00:20:15.654 --> 00:20:21.474 So we build models for that to accelerate the learning about the primary endpoint. 00:20:21.624 --> 00:20:23.094 We haven't changed the endpoint. 00:20:23.884 --> 00:20:28.734 The endpoint is six-month pathologic complete response, but early 00:20:28.734 --> 00:20:33.704 values are driving that estimate, which are driving the adaptation 00:20:36.064 --> 00:20:40.454 Now, an important part of, uh, you know, what does that model look like? 00:20:40.454 --> 00:20:43.844 And I'll come back to specifically what that model looks like 00:20:44.164 --> 00:20:46.014 and how we learn from that. 00:20:46.794 --> 00:20:49.424 That's the aspect in adaptive trials. 00:20:49.424 --> 00:20:57.144 So what-- when people ask me what scenarios, what trials can benefit from 00:20:57.144 --> 00:21:03.654 adaptive design, the most important part of it is that speed of learning. 00:21:05.164 --> 00:21:11.584 So if you have a trial where the endpoint, uh, i- is a migraine trial where it's 00:21:11.584 --> 00:21:15.344 a treatment of acute migraine, there are, there are treatments that are 00:21:15.344 --> 00:21:20.554 trying to prevent migraines, but suppose it's the treatment of acute migraine. 00:21:20.554 --> 00:21:23.994 A lot of times we look at two hours later, are you pain-free? 00:21:25.344 --> 00:21:29.094 So when you give a treatment to a patient, you know the outcome 00:21:29.094 --> 00:21:31.734 of that patient two hours later. 00:21:33.254 --> 00:21:39.124 We can do a lot of really efficient things adaptively because the, the, the 00:21:39.124 --> 00:21:47.908 learning ratio is really fast, and so we can be very efficient Alternatively, if 00:21:47.908 --> 00:21:54.978 the endpoint doesn't come for 18 months in that scenario, we've got a whole lot 00:21:54.978 --> 00:21:59.148 of patients that have been enrolled that we don't know anything about, and we're 00:21:59.148 --> 00:22:01.368 waiting for them to get to 18 months. 00:22:01.598 --> 00:22:03.568 It's hard for the trial to be efficient. 00:22:03.908 --> 00:22:05.368 Should we stop enrolling? 00:22:05.368 --> 00:22:07.138 Should we change doses? 00:22:07.408 --> 00:22:08.858 Should we make changes? 00:22:09.348 --> 00:22:11.268 We don't know anything at that point. 00:22:12.018 --> 00:22:14.448 So can we speed that up? 00:22:14.898 --> 00:22:20.628 So adaptive trials become more efficient relative to fixed trials 00:22:21.388 --> 00:22:27.368 the faster we can accelerate that learning, and that becomes a huge part 00:22:27.368 --> 00:22:33.908 of adaptive trials, and it becomes the huge fight that we have with time. 00:22:33.908 --> 00:22:37.928 We're fighting time, in that case, to make the trial more 00:22:37.928 --> 00:22:41.398 efficient, to treat patients better, to learn more efficiently. 00:22:41.768 --> 00:22:44.638 That's the whole promise of adaptive trials. 00:22:44.938 --> 00:22:51.058 A huge part of building adaptive trials is this acceleration of learning within 00:22:51.058 --> 00:22:53.038 single patients that are moving through. 00:22:54.238 --> 00:22:58.588 It's what w- what I've been doing for 26 years. 00:22:59.028 --> 00:23:04.318 Now, the interesting thing about it, a lot of fixed trials don't care about this. 00:23:05.498 --> 00:23:11.618 If your primary endpoint is 12 months, and you're gonna enroll 200 patients, 00:23:11.618 --> 00:23:15.308 and you're gonna wait till everybody gets to 12 months, you don't really 00:23:15.308 --> 00:23:21.018 care if six months is predictive of 12, if three months is predictive. 00:23:21.278 --> 00:23:24.058 It's irrelevant to the primary analysis. 00:23:24.408 --> 00:23:30.348 Now, you might do an MMRM model that incorporates that, but it's all done when 00:23:30.398 --> 00:23:35.498 all patients have had the opportunity to complete the primary endpoint time. 00:23:35.878 --> 00:23:40.748 Now, the whole world of this is missing data, but this is 00:23:40.748 --> 00:23:42.908 different kind of missing data. 00:23:43.188 --> 00:23:46.458 These are patients that just haven't reached that time point, which is a 00:23:46.458 --> 00:23:53.168 whole new thing in adaptive trials that don't come into it in fixed trials. 00:23:54.308 --> 00:24:00.848 So this whole thing of trying to accelerate the learning is new and 00:24:00.848 --> 00:24:04.128 relatively specific to adaptive trials. 00:24:05.458 --> 00:24:10.928 Okay, so a lot of what we do is spending this time on building 00:24:10.928 --> 00:24:13.238 that predictive of it in it. 00:24:13.288 --> 00:24:15.928 So thinking about fighting time here. 00:24:15.958 --> 00:24:17.908 Let's go back to I-SPY 2. 00:24:19.520 --> 00:24:23.680 So in I-SPY 2, this-- what, what does this model look like? 00:24:24.750 --> 00:24:31.870 At one month and three months and six months, we get an observation 00:24:32.030 --> 00:24:38.850 of the, the, the size of the tumor relative to baseline. 00:24:39.650 --> 00:24:42.230 And let's think about percent reduction. 00:24:42.820 --> 00:24:49.490 So the endpoint that we get at one month and three months is a percent reduction. 00:24:51.050 --> 00:24:53.610 We could get 100% reduction. 00:24:53.610 --> 00:24:55.950 We could get 50% reduction in that. 00:24:57.100 --> 00:25:03.680 Now, we're interested in suppose somebody at one month, a, a woman at one month 00:25:04.120 --> 00:25:09.150 has an 80% reduction in tumor size. 00:25:10.670 --> 00:25:14.940 What does that mean about the likelihood of pathologic complete response? 00:25:16.910 --> 00:25:22.550 That's, that's really what we're interested in, uh, from that perspective. 00:25:23.460 --> 00:25:29.940 The beauty of the trial itself is that patients pass through 00:25:30.120 --> 00:25:31.860 one month and six months. 00:25:32.550 --> 00:25:38.220 We get to see a lot of observations on women that went from one month and 00:25:38.220 --> 00:25:43.480 had an MRI, and then we found out were they a pathologic complete response. 00:25:44.310 --> 00:25:50.940 So we get a lot of pairs of what was the percent reduction at one month, and 00:25:50.940 --> 00:25:53.710 were they a responder at six months? 00:25:54.140 --> 00:25:57.100 So we're building a model here for that prediction. 00:25:57.100 --> 00:26:02.320 You could build a logistic regression model for that as that percent 00:26:02.320 --> 00:26:05.310 reduction to a response yes/no. 00:26:06.630 --> 00:26:13.870 Now, we built that original model, and a lot of times we're building that model 00:26:13.870 --> 00:26:19.240 in adaptive trial, and we don't have the data in the middle of the trial. 00:26:19.240 --> 00:26:25.980 We, we don't get the luxury of opening up the data and building 00:26:25.980 --> 00:26:28.990 a model and say, "Oh, this is what the behavior of the model. 00:26:28.990 --> 00:26:30.520 This is the best-fitting model. 00:26:30.520 --> 00:26:36.110 Here's the BIC," you know, whatever, whatever modeling approach you take. 00:26:36.110 --> 00:26:36.700 We don't have it. 00:26:36.700 --> 00:26:38.700 We build it before the trial starts. 00:26:38.750 --> 00:26:39.950 It's hard-coded. 00:26:40.300 --> 00:26:44.270 It runs the model at, at that time point, so we have to 00:26:44.270 --> 00:26:45.730 build the model ahead of time. 00:26:46.480 --> 00:26:50.170 We had the luxury in I-SPY 2 of actually I-SPY 1. 00:26:50.780 --> 00:26:53.300 I-SPY 1 was an observational cohort. 00:26:53.300 --> 00:26:56.820 It didn't, it didn't change the treatment of patients, but it got 00:26:56.820 --> 00:27:01.030 these observations, and so we had about 40 patients where we, we 00:27:01.030 --> 00:27:02.990 could build a tri-- build a model. 00:27:03.410 --> 00:27:09.444 Now, we want that model to be robust 00:27:11.582 --> 00:27:18.162 We don't want it to be overly strong in ways that it's making bad forecasts 00:27:18.162 --> 00:27:25.032 of PCR because it's somewhat hard coded and it's running that and it doesn't 00:27:25.032 --> 00:27:29.242 necessarily know it's not fitting well, and then it's driving decisions 00:27:29.242 --> 00:27:34.382 about allocation to women that way with breast cancer based on a model that 00:27:34.422 --> 00:27:36.462 maybe isn't, isn't fitting very well. 00:27:37.232 --> 00:27:39.942 So we want this to be really robust. 00:27:40.512 --> 00:27:47.722 So the model we built in that circumstance is a piecewise 00:27:50.240 --> 00:27:58.320 prediction model where we actually had 13 pieces to the model that then 00:27:58.320 --> 00:28:03.500 forecasts the relative odds of a PCR. 00:28:04.250 --> 00:28:05.410 What do I mean by that? 00:28:05.950 --> 00:28:12.770 We broke that interval up into suppose the reduction is in the-- is zero to 10%. 00:28:13.520 --> 00:28:14.330 That's an outcome. 00:28:14.780 --> 00:28:16.260 By the way, the tumor could grow. 00:28:16.330 --> 00:28:18.230 That's the, that's the worst outcome. 00:28:19.210 --> 00:28:25.570 Then it could be at zero to 10, 10 to 20, 20 to 30, 30 to 40, and 00:28:25.570 --> 00:28:32.470 so on, all the way down up to 80 to 90, and then we break 90 to 95. 00:28:32.470 --> 00:28:37.450 Remember, pathologic complete response is six months, there's zero tumor. 00:28:38.360 --> 00:28:43.370 So 90 to 95, 95 to 99, and then more than 99. 00:28:43.880 --> 00:28:51.630 Those are the 13 classifications of the MRI at one month, and then a separate 00:28:51.670 --> 00:28:56.730 instance of the model is at three months, and a separate instance is at six months. 00:28:57.910 --> 00:29:07.000 Then we learn that relative, uh, the relative odds of a responder as a 00:29:07.000 --> 00:29:10.990 function of which category your MRI is in. 00:29:12.860 --> 00:29:17.310 And based on that, that's learned from past women in the trial. 00:29:17.340 --> 00:29:21.290 It's one of the beautiful things about a platform trial is we have 00:29:21.410 --> 00:29:27.600 a lot of data on that in ICE by two from previous arms and previous… 00:29:27.600 --> 00:29:33.654 the, the control throughout that So that model is running. 00:29:33.854 --> 00:29:40.364 So at the time, let's go back to this time where one patient had gone 00:29:40.364 --> 00:29:42.374 through, and this estimate was 60%. 00:29:44.094 --> 00:29:51.304 We had a number of women at that time in the trial, and I'm gonna try to remember. 00:29:51.304 --> 00:29:57.704 I believe we had some 60-plus women who were on the arm that 00:29:57.704 --> 00:30:03.654 had been allocated at that time, but only one had passed through. 00:30:05.144 --> 00:30:14.134 Now, all of the other women in the trial have some level of, of maturity of their, 00:30:14.244 --> 00:30:16.514 their MRI data, one month, three month. 00:30:16.714 --> 00:30:19.184 Some have-- don't, haven't even reached one month, and they 00:30:19.184 --> 00:30:24.634 don't inform the, the l- the, the response, the PCR six-month response. 00:30:25.384 --> 00:30:27.974 And then we have some that have six months but haven't had 00:30:27.974 --> 00:30:29.574 surgery yet at that time point. 00:30:30.484 --> 00:30:34.094 So when this panic happened, of course, we jumped in and we're, 00:30:34.094 --> 00:30:35.534 we're looking at the model. 00:30:35.814 --> 00:30:40.394 You want humans to, to watch this automatic pilot to make sure that 00:30:40.394 --> 00:30:46.804 if, if the model is broken to some extent, that w- we, we make sure 00:30:46.804 --> 00:30:48.364 it's doing what it's supposed to do. 00:30:48.394 --> 00:30:50.564 So we went in and we investigated. 00:30:51.264 --> 00:30:55.844 And as Don says, pembrolizumab was just melting the tumor away. 00:30:56.374 --> 00:31:01.534 The values of the MRI for that treatment were really, really good. 00:31:01.884 --> 00:31:09.614 Lots of them were 99% reduction, 95 to 99% reduction, that the model was 00:31:09.614 --> 00:31:14.444 predicting that a lot of these were going to be six-month PCRs, and the 00:31:14.444 --> 00:31:20.784 fraction of them that are going to be six-month PCRs was 60% out of them. 00:31:21.114 --> 00:31:24.464 It was all based on the longitudinal model. 00:31:24.494 --> 00:31:25.334 We had one patient. 00:31:26.014 --> 00:31:32.580 It was entirely driven by the longitudinal prediction of those patients Uh, by the 00:31:32.580 --> 00:31:34.520 way, at that time, I, I should back up. 00:31:34.520 --> 00:31:37.520 Some 60 patients had been enrolled with pembrolizumab. 00:31:38.420 --> 00:31:45.500 The I-SPY2 trial had these different, uh, categories that… 00:31:45.600 --> 00:31:52.480 Uh, different subtypes that are differentiated largely by HER2 00:31:52.480 --> 00:31:54.670 status and hormone receptor status. 00:31:55.610 --> 00:32:00.220 One of those subtypes is called triple-negative breast cancer, and 00:32:00.220 --> 00:32:07.070 that's that you're negative on hor- uh, a hormone receptor status and HER2 status. 00:32:07.100 --> 00:32:13.270 There were 29 of those patients that had been enrolled, and one of those 00:32:13.270 --> 00:32:15.850 had passed through the six months. 00:32:16.490 --> 00:32:22.450 That was the subtype that was estimated to be a 60% response. 00:32:23.570 --> 00:32:29.990 At the time that it said, "Graduate this, stop enrolling, and pembrolizumab should 00:32:29.990 --> 00:32:35.180 go to Phase 3," the final analysis takes place when all patients get to six months. 00:32:35.770 --> 00:32:37.160 It's another really… 00:32:37.400 --> 00:32:43.360 As statisticians, it's just ano- a- another really, um, neat, exciting 00:32:43.760 --> 00:32:46.720 area where we're doing these forecasts. 00:32:46.720 --> 00:32:51.630 It's forecasts and predictions of what's gonna happen, and you stop 00:32:51.630 --> 00:32:55.550 enrolling a treatment arm, and then you get to see the outcome of it. 00:32:56.410 --> 00:32:59.440 So you're doing these predictions, but you get to observe the outcome. 00:32:59.700 --> 00:33:02.730 In many circumstances, we might make predictions, and we never actually 00:33:02.730 --> 00:33:06.040 get to see the outcome of that, but it's forecasting what would happen. 00:33:06.250 --> 00:33:08.230 Here, we get to see the outcome of that. 00:33:09.310 --> 00:33:11.910 So there was nothing wrong with the model. 00:33:12.350 --> 00:33:15.290 It was functioning exactly as designed. 00:33:15.620 --> 00:33:20.210 Made some people nervous that it was this, this forecast entirely. 00:33:20.890 --> 00:33:22.080 Well, they… 00:33:22.540 --> 00:33:29.210 The pembrolizumab stopped enrolling, and the trial read out, and it was published 00:33:29.210 --> 00:33:41.800 in JAMA Oncology, and the final estimate for the PCR response rate was 60%. 00:33:43.360 --> 00:33:49.870 It actually, the, the final posterior estimate of that, it, it triggered 00:33:49.870 --> 00:33:52.880 it almost exactly in the setting. 00:33:53.390 --> 00:33:54.690 Really, uh, uh… 00:33:54.730 --> 00:33:59.010 And, and by the way, it was an over 99% probability of winning 00:33:59.010 --> 00:34:03.670 Phase 3 trial in triple-negative breast cancer, and of course, 00:34:03.670 --> 00:34:09.960 pembrolizumab is an amazing therapy now, um, uh, in many, many different 00:34:10.680 --> 00:34:15.488 cancers So it actually hit exactly. 00:34:15.608 --> 00:34:19.978 Uh, ended up 60% to a control rate of 22%, and you can go see 00:34:19.978 --> 00:34:25.248 the posterior distribution in that paper, uh, Nanda et al., 00:34:25.508 --> 00:34:27.028 and, uh, within that. 00:34:27.058 --> 00:34:34.868 So it triggered it perfectly six months ahead of what if we'd have waited for 00:34:34.868 --> 00:34:38.208 all of those patients, and meanwhile, we would've enrolled more in that. 00:34:38.538 --> 00:34:43.358 Now, in the design you wanna set up, what do we want the trial to do? 00:34:43.358 --> 00:34:44.828 Do we want it to do that? 00:34:44.828 --> 00:34:49.048 These are all decisions made before the arm started in the trial 00:34:51.282 --> 00:34:53.582 Okay, so accelerating it. 00:34:53.582 --> 00:34:55.262 That's a case where even six months… 00:34:55.562 --> 00:35:01.172 By the way, a, a huge problem in breast cancer is there are really many effective 00:35:01.172 --> 00:35:05.922 therapies looking at endpoints like overall survival, event-free survival. 00:35:06.222 --> 00:35:10.322 These endpoints are, are thankfully very, very long. 00:35:10.952 --> 00:35:14.542 To do trials and learning trials in an adjuvant setting 00:35:14.542 --> 00:35:18.422 looking at those endpoints are really, really hard and long. 00:35:18.692 --> 00:35:24.422 And so a huge desire in, in many of these diseases are looking at endpoints 00:35:24.422 --> 00:35:26.312 that are earlier that are predictive. 00:35:26.732 --> 00:35:30.292 That's a different part of disease modeling and longitudinal 00:35:30.292 --> 00:35:32.112 modeling and hugely important. 00:35:33.252 --> 00:35:36.172 And that, that, that's a separate thing. 00:35:36.172 --> 00:35:39.382 Look for surrogate markers and huge thing. 00:35:39.382 --> 00:35:44.542 And Barry does a lot of that modeling in other diseases and, and cancer. 00:35:44.882 --> 00:35:47.952 But this is the setting where let's sort of fix in that the 00:35:47.952 --> 00:35:51.752 trial has a primary endpoint, and within that trial, we're trying to 00:35:51.752 --> 00:35:53.842 accelerate to that primary endpoint 00:35:56.382 --> 00:35:56.862 Okay. 00:35:57.832 --> 00:36:00.782 What are other examples of this? 00:36:00.782 --> 00:36:05.712 What are other examples of fighting time in clinical trials? 00:36:06.842 --> 00:36:12.872 Uh, I'll go back to, uh, BAN2401 adaptive trial for Alzheimer's. 00:36:13.662 --> 00:36:16.622 This is now the treatment is lecanemab. 00:36:16.742 --> 00:36:21.932 It's, it was the first disease-modifying treatment approved for Alzheimer's. 00:36:21.932 --> 00:36:25.752 This is Eisai, uh, ran the development of this. 00:36:26.132 --> 00:36:34.472 They ran a Phase 2 trial, and in the Phase 2 trial, they had five doses and control. 00:36:34.902 --> 00:36:40.552 The doses were different dose levels and frequency, monthly 00:36:40.552 --> 00:36:42.462 or bi-monthly infusions. 00:36:43.182 --> 00:36:48.212 And they ran a, an adaptive trial that included response adaptive 00:36:48.212 --> 00:36:50.442 randomization over the different doses. 00:36:51.102 --> 00:36:56.432 Whichever doses were doing-- were, were, were working better, they wanted 00:36:56.432 --> 00:37:01.202 to put more patients on that to learn what is the right dose going forward. 00:37:01.962 --> 00:37:04.222 They wanted proof of concept. 00:37:04.222 --> 00:37:08.272 If, if the predictive probability of winning Phase 3 got high enough, 00:37:08.302 --> 00:37:10.232 they wanted to jump to Phase 3. 00:37:10.562 --> 00:37:12.372 They could stop for futility. 00:37:12.862 --> 00:37:17.102 The trial had an adaptive sample size from three hundred to eight hundred. 00:37:17.702 --> 00:37:22.532 The primary endpoint in the trial was 18 months in that. 00:37:23.242 --> 00:37:29.632 Now, to make all these decisions in the trial, it's hard to do a response 00:37:29.632 --> 00:37:33.652 adaptive randomization if you have to wait for patients to get to 18 months. 00:37:34.712 --> 00:37:40.822 Now, the primary endpoint in that trial was ADCOMS, which is a cognitive measure. 00:37:40.822 --> 00:37:47.322 It's actually a combination of different, uh, questions from CDR sum of boxes, 00:37:47.322 --> 00:37:53.032 ADAS-Cog, and MMSE, and they were, uh, selected to be earlier in the disease 00:37:53.032 --> 00:37:57.282 course, cognitive-type endpoints, that would be more informative for 00:37:57.322 --> 00:38:01.072 early Alzheimer's disease, which is what was done in this trial. 00:38:03.400 --> 00:38:08.720 In order to be an efficient design, we needed to learn faster than 18 months. 00:38:09.230 --> 00:38:13.860 We get three, six, nine, 12, 15, 18 months values. 00:38:13.860 --> 00:38:16.000 Every three months, we get patients' data. 00:38:16.810 --> 00:38:24.810 All of that data was used to forecast the effect of the different doses relative 00:38:24.810 --> 00:38:30.840 to control, estimate the control rate, to, to speed that learning up, to have 00:38:30.870 --> 00:38:33.470 all of those adaptations take place. 00:38:34.750 --> 00:38:41.260 The prediction-- So we use the same endpoint observed at earlier time periods. 00:38:42.350 --> 00:38:43.510 They're not primary. 00:38:43.510 --> 00:38:48.930 They're the, they, they're being used as an auxiliary endpoint, a predictive 00:38:48.930 --> 00:38:55.420 endpoint for the final cognitive outcome for a patient, which is 18 months. 00:38:57.000 --> 00:39:01.720 Now, in this trial, we did, uh, largely the early part of the 00:39:01.720 --> 00:39:05.980 trial was predicting 12 months, but the full exposure went out to 18. 00:39:06.240 --> 00:39:09.080 So I'm gonna alter that a little bit and talk about 12 months. 00:39:09.520 --> 00:39:12.790 The, the adaptive algorithm was driven by the 12 months. 00:39:13.890 --> 00:39:20.620 Now, the, the model we used was a linear regression model between three months and 00:39:20.620 --> 00:39:27.360 12 months, six months and 12 months, nine months and 12 months, informed by previous 00:39:27.360 --> 00:39:31.630 patients that had path-- passed through the trial, passed through those endpoints 00:39:31.700 --> 00:39:38.840 in the trial itself, and we had previous data on the expected correlation of those. 00:39:39.220 --> 00:39:43.690 We used prior distributions for those correlations from previous data, 00:39:44.430 --> 00:39:50.830 but then that's updated in the trial itself on that correlation to then 00:39:50.830 --> 00:39:53.060 drive the estimates of the treatment. 00:39:53.920 --> 00:40:00.170 It's a beautiful, uh, uh, example, in part because ASID published the results 00:40:00.170 --> 00:40:02.560 of every one of the interim analyses. 00:40:03.190 --> 00:40:08.720 You can go in and see what did it forecast at the first interim analysis 00:40:08.720 --> 00:40:10.620 of the trial, which was 200 patients. 00:40:11.120 --> 00:40:15.950 And then every 50 patients in the trial, it did an, it did an, an update. 00:40:16.660 --> 00:40:22.760 At that first interim analysis, which is triggering 12 months, there 00:40:22.760 --> 00:40:24.676 were zero patients at 12 months. 00:40:26.366 --> 00:40:30.406 At the second interim analysis at 250, I think we had several 00:40:30.406 --> 00:40:31.856 that had reached 12 months. 00:40:32.316 --> 00:40:35.696 A good number have gotten to nine months, which is really predictive 00:40:35.696 --> 00:40:39.276 of 12 months and six months, so they're progressing through that. 00:40:39.856 --> 00:40:43.936 Response adaptive randomization started at that 200. 00:40:44.586 --> 00:40:49.576 At that time, the response adaptive randomization immediately wanted the two 00:40:49.576 --> 00:40:57.146 high doses, 10 monthly and 10 bimonthly, and kind of went away from the lower doses 00:40:57.146 --> 00:40:59.856 based on cognitive values at three… 00:40:59.856 --> 00:41:01.356 mostly three and six months. 00:41:02.666 --> 00:41:07.516 You-- The randomization probabilities change, then the data get updated 50 00:41:07.516 --> 00:41:11.806 patients later, and the exposure of the early patients gets longer, which is 00:41:11.806 --> 00:41:17.486 critical in adaptive design because that's the most valuable data in the trial. 00:41:18.366 --> 00:41:20.466 The values were updated again. 00:41:21.436 --> 00:41:26.896 Interestingly in the trial, the final sample sizes of this, by the time 00:41:26.896 --> 00:41:33.016 everybody got to 18 months, were about 250 patients on the control, placebo. 00:41:33.826 --> 00:41:40.656 About 250 patients on 10 monthly and about 175 on 10 bimonthly. 00:41:42.106 --> 00:41:46.466 If you add together the other three doses, it was about 175. 00:41:46.546 --> 00:41:51.556 So these two doses got the lion's share of patients, appropriately so. 00:41:52.086 --> 00:41:58.516 The 10 bimonthly arm of, eventually is the w- arm that went to the phase three 00:41:58.516 --> 00:42:00.906 trial and demonstrated superiority. 00:42:02.136 --> 00:42:05.496 About a 27% slowing of disease. 00:42:06.106 --> 00:42:12.096 Interestingly, in the adaptive trial of 800 patients, if you go through the 00:42:12.096 --> 00:42:21.636 movie of what happened, that forecast was about, uh, interim five or six or 00:42:21.636 --> 00:42:29.266 seven, right in there, the algorithm started to estimate a 25% to 30% slowing 00:42:29.996 --> 00:42:34.216 and probability of benefit above 95%. 00:42:35.556 --> 00:42:41.716 The probability of at least a 25% slowing, which is what the criteria was in the 00:42:41.716 --> 00:42:48.786 trial, that was what they wanted to define go, at interim five was about 75%. 00:42:49.316 --> 00:42:52.886 Seventy-five percent chance that you have at least a 25% slowing. 00:42:53.636 --> 00:43:02.076 The final number when all patients got through 18 months, years later, was 76%. 00:43:04.040 --> 00:43:07.670 Now, there was the huge value of the follow-up and all of that, but 00:43:07.670 --> 00:43:09.540 the algorithm was forecasting that. 00:43:09.540 --> 00:43:14.000 Now it's forecasting it by doses and place pacing-- placing patients on 00:43:14.000 --> 00:43:19.360 the right doses, and it made for a much more efficient trial design. 00:43:19.860 --> 00:43:23.550 It enabled them to explore five doses by the ability to do 00:43:23.550 --> 00:43:25.220 response adaptive randomization. 00:43:25.840 --> 00:43:31.110 But none of that happens without the longitudinal modeling of the endpoint. 00:43:31.400 --> 00:43:35.680 You learn nothing until it's too late to make those adaptations. 00:43:37.580 --> 00:43:38.000 Okay. 00:43:39.220 --> 00:43:44.990 A couple other examples of this, just to give you the wide view of, 00:43:44.990 --> 00:43:49.390 of what this looks like, different trial designs utilizing this. 00:43:49.810 --> 00:43:55.160 The Trulicity is El-Eli Lilly's… 00:43:55.160 --> 00:44:02.890 It was a GLP-1, uh, kinda kicked off Eli Lilly's, um, um, jumping into to GLP-1s, 00:44:02.890 --> 00:44:08.070 which, uh, I assume everybody understands the story of the impact of these. 00:44:08.320 --> 00:44:13.670 This was treatment of diabetes, and they ran a seamless two-three 00:44:13.670 --> 00:44:16.100 trial, the Award Five trial. 00:44:16.670 --> 00:44:18.910 You can go see the publications of that. 00:44:19.250 --> 00:44:24.540 And in the seamless, in the phase two part of it, they were exploring seven 00:44:24.540 --> 00:44:28.820 different doses of, of dulaglutide. 00:44:28.820 --> 00:44:34.760 Uh, uh, the trade name became Trulicity after it was eventually approved. 00:44:35.440 --> 00:44:38.470 The seven different doses were explored. 00:44:38.910 --> 00:44:41.410 The primary endpoint was HbA1c. 00:44:41.820 --> 00:44:47.330 Interestingly, there was a utility function over four endpoints where they 00:44:47.330 --> 00:44:52.260 wanted to make the decision on allocating the doses and dose selection for phase 00:44:52.260 --> 00:45:02.302 three, taking into account HbA1c, HbA1c change uh, blood pressure and heart rate. 00:45:02.772 --> 00:45:06.252 So they didn't want those things to be elevated to the 00:45:06.252 --> 00:45:08.422 point of cardiovascular risk. 00:45:09.212 --> 00:45:11.672 The fourth endpoint was weight loss. 00:45:12.342 --> 00:45:17.852 The more weight loss, the more benefit it was perceived, the more, the, the better 00:45:17.852 --> 00:45:22.382 the therapeutic profile and the better patients would want to take the treatment. 00:45:22.672 --> 00:45:30.022 If there was weight gain, there was no commercial en- aspect of this treatment. 00:45:30.652 --> 00:45:35.382 Ironically, that became the huge part of GLP-1s are now approved specifically 00:45:35.382 --> 00:45:39.732 for weight loss, even in non-diabetics, and that played a huge role. 00:45:39.732 --> 00:45:43.152 In that trial, that was one of four endpoints in a utility function. 00:45:44.082 --> 00:45:49.102 Over the seven endpoints, response adaptive randomization was done, the 00:45:49.102 --> 00:45:55.162 ability to jump from phase two to phase three, and it was predicting will it 00:45:55.162 --> 00:46:02.682 show non-inferiority to sitagliptin as an active comparator for the primary 00:46:02.682 --> 00:46:09.834 endpoint of the eventual move to phase three It made that tw- that's a 12-month 00:46:09.834 --> 00:46:15.744 endpoint, and during the trial we're enrolling and we're getting monthly 00:46:15.744 --> 00:46:22.914 values on HbA1c change, and we built a model between those values and 12 months. 00:46:23.914 --> 00:46:25.934 It jumped to phase three. 00:46:25.934 --> 00:46:29.404 It selected two doses, 1.5 00:46:29.404 --> 00:46:30.644 milligram and 0.75 00:46:30.644 --> 00:46:36.614 milligrams, to go to phase three, and there were no patients at 12 months. 00:46:37.314 --> 00:46:43.564 It was entirely based on the longitudinal modeling of HbA1c change, 00:46:43.834 --> 00:46:48.284 which was pretty well understood, the behavior of that endpoint. 00:46:48.284 --> 00:46:51.554 They had previous data on patients, and again, we spent a 00:46:51.554 --> 00:46:53.094 lot of time building that model. 00:46:53.344 --> 00:46:57.304 Interestingly, early on, the question was, what's a better 00:46:57.304 --> 00:47:00.424 predictor of 12-month HbA1c? 00:47:01.464 --> 00:47:07.364 Early values of HbA1c or fasting blood glucose, which is a measure 00:47:07.364 --> 00:47:11.204 that is thought to move faster, and maybe that's a better predictor. 00:47:11.264 --> 00:47:14.114 We actually had data to jump in and look at if we were gonna 00:47:14.114 --> 00:47:18.474 predict 12 months which one was better, and HbA1c was better. 00:47:19.114 --> 00:47:23.374 It's thought that that's kind of an integrated value over time, and maybe 00:47:23.374 --> 00:47:28.624 it's slow-moving, but it was, was much better predictor of 12 months 00:47:28.624 --> 00:47:30.314 than fasting blood glucose was. 00:47:31.074 --> 00:47:35.494 And so these decisions in the building of that trial all go in because 00:47:35.494 --> 00:47:36.864 we're trying to accelerate it. 00:47:37.814 --> 00:47:43.574 Now, Lilly did something tremendous in that trial, and I've not seen it 00:47:43.574 --> 00:47:49.144 in another trial, is they knew that this longitudinal model was a really 00:47:49.144 --> 00:47:52.264 important part to selecting a dose. 00:47:52.264 --> 00:47:58.974 That dose that went to phase 3 spawned cardiovascular trials, other trials. 00:47:59.294 --> 00:48:01.224 That was the decision on dose. 00:48:01.294 --> 00:48:04.494 It was making that drug development decision. 00:48:04.864 --> 00:48:05.874 The 1.5 00:48:05.874 --> 00:48:10.964 milligram dose became the dose that was used for dulaglutide and has 00:48:10.964 --> 00:48:13.514 made billions for, for Eli Lilly. 00:48:14.214 --> 00:48:19.564 And so that key decision, they didn't want to enroll too fast. 00:48:19.564 --> 00:48:22.474 Remember, this whole thing is about time to information. 00:48:23.044 --> 00:48:28.004 How quickly do we learn to make good decisions like a seamless decision to 00:48:28.004 --> 00:48:34.580 go to phase 3 They controlled enrollment We simulated the trial for a range of 00:48:34.580 --> 00:48:40.650 enrollments, and we saw that if you enroll too fast, you pile patients in, it's less 00:48:40.650 --> 00:48:49.040 efficient than enrolling slower to allow for further follow-up and, and, and data. 00:48:49.340 --> 00:48:50.900 We were making better decisions. 00:48:50.900 --> 00:48:56.080 It led to a more efficient development strategy to go slower. 00:48:56.560 --> 00:49:01.360 And they were worried if they handed it over that, that operationally 00:49:01.360 --> 00:49:03.180 we wanna enroll fast, fast, fast. 00:49:03.660 --> 00:49:08.510 They knew that slower was actually more efficient of a development strategy. 00:49:08.930 --> 00:49:11.260 And they controlled it, and it went there. 00:49:11.590 --> 00:49:14.730 It jumped to phase three as soon as it could. 00:49:14.730 --> 00:49:18.510 The data were amazing for the treatment, and it showed superiority 00:49:18.510 --> 00:49:20.890 to sitagliptin on HbA1c change. 00:49:21.420 --> 00:49:27.956 Um, and they say it s- accelerated development 12 to 18 months The 00:49:27.956 --> 00:49:32.586 key to that was the longitudinal modeling of HbA1c and weight loss, 00:49:32.666 --> 00:49:36.886 the prediction of six-month weight loss based on earlier values, which 00:49:36.886 --> 00:49:38.646 is very, very predictive, by the way. 00:49:39.996 --> 00:49:43.376 Okay, so also a really nice example of that. 00:49:43.886 --> 00:49:46.696 We, we do a lot of stroke trials. 00:49:46.786 --> 00:49:51.296 Uh, ICE CAP was the topic of a podcast last week where the 00:49:51.296 --> 00:49:53.856 endpoint is 90-day modified Rankin. 00:49:54.586 --> 00:50:01.226 It's an ordinal scale, zero through six, of their status at 90 days. 00:50:02.506 --> 00:50:07.466 In the, in the ICE CAP trial, we use their 30-day modified Rankin 00:50:07.466 --> 00:50:12.556 status as a predictor of 90 days, and it's an incredibly good predictor. 00:50:13.296 --> 00:50:17.156 Interestingly, by the way, if you go back to the ICE by two example, I know 00:50:17.156 --> 00:50:22.206 I'm jumping around here a bit, the prediction of the MRIs, what was really 00:50:22.206 --> 00:50:30.156 interesting is the most predictive thing is if the reduction wasn't very great, 00:50:30.546 --> 00:50:32.756 if it was less than a 50% reduction. 00:50:32.756 --> 00:50:34.556 At, at one month, that wasn't too bad. 00:50:34.556 --> 00:50:39.736 But if it was a 25 or 30% reduction, very unlikely you're gonna be a PCR. 00:50:40.536 --> 00:50:41.706 PCR is a high hurdle. 00:50:42.246 --> 00:50:47.016 If you weren't seeing 50%-plus reduction at the first month and 00:50:47.016 --> 00:50:50.236 90-plus at the third, you were unlikely. 00:50:50.806 --> 00:50:52.726 If you were seeing 90 per… 00:50:52.726 --> 00:50:57.916 90 to 95, it was kind of 50/50 whether you're gonna get there. 00:50:57.916 --> 00:51:00.806 If you're seeing 99%-plus, then you're highly likely. 00:51:01.496 --> 00:51:05.666 Some of the most predictive part in that trial was the lack of response, 00:51:06.016 --> 00:51:11.666 the lack of MR-- good MRIs became very, very predictive, where other 00:51:11.666 --> 00:51:16.266 values are less predictive but valuable that it's not negative. 00:51:17.306 --> 00:51:23.236 In MRS, for example, a, a 90-- a 30-day MRS of six means the 00:51:23.236 --> 00:51:27.706 patient's dead is a really strong predictor of death at 90 days. 00:51:28.096 --> 00:51:31.476 Some of the middling values can bounce around a little bit, but it's 00:51:31.476 --> 00:51:33.786 incredibly predictive of 90 days. 00:51:34.116 --> 00:51:40.646 You can accelerate your adaptive design two months by using 30-day values. 00:51:41.216 --> 00:51:43.236 Now, how do you do this? 00:51:43.236 --> 00:51:48.816 Well, we incorporate Bayesian models, not surprisingly, uh, within it. 00:51:48.816 --> 00:51:52.366 They're really good for longitudinal models because they incorporate 00:51:52.366 --> 00:51:58.430 uncertainty really, really well They can use prior models, prior 00:51:58.430 --> 00:51:59.990 estimates of that correlation. 00:51:59.990 --> 00:52:03.260 So you're doing analyses with, with little data early on. 00:52:03.580 --> 00:52:06.140 You want it to be smart in that circumstance. 00:52:06.880 --> 00:52:10.600 So what we do is we build a model, and we have a distribution of the 00:52:10.600 --> 00:52:12.900 parameters of the longitudinal model. 00:52:13.610 --> 00:52:17.620 That's updated through Bayesian modeling, Markov chain, Monte Carlo. 00:52:18.190 --> 00:52:22.560 And then we make a forecast of every patient that's early in the trial. 00:52:23.150 --> 00:52:25.930 Based on that longitudinal model makes a forecast. 00:52:26.630 --> 00:52:33.540 We, we multiply impute those values, which gives you a complete set of data. 00:52:33.540 --> 00:52:37.320 Every patient has a final v-value for that. 00:52:38.240 --> 00:52:43.750 Then we update the final analysis modeling because we have a complete trial, and 00:52:43.750 --> 00:52:45.770 then we do that again and again and again. 00:52:45.770 --> 00:52:51.990 So at every interim, we're simulating ten thousand complete trials 00:52:51.990 --> 00:52:55.520 where that imputation is done through the longitudinal modeling. 00:52:56.220 --> 00:52:59.620 Become a very common technique now for missing data. 00:53:00.010 --> 00:53:02.350 Very valuable technique for missing data. 00:53:02.670 --> 00:53:08.210 That's the similar stuff we're doing for an a-adaptive trial in the middle of it. 00:53:08.480 --> 00:53:12.110 So the fun part of this, we do a lot of this with different kinds 00:53:12.110 --> 00:53:16.380 of endpoints, different kinds of predictors w-within the trial. 00:53:16.380 --> 00:53:20.560 We get to build these models from them, this ordinal model 00:53:20.590 --> 00:53:23.220 of thirty days to ninety days. 00:53:23.850 --> 00:53:28.650 If you're a three, how do we predict values at ninety days? 00:53:29.350 --> 00:53:31.890 If you're a four, how do we predict values? 00:53:32.340 --> 00:53:36.470 Do we want those forecasts only to look at the patients that were threes? 00:53:37.576 --> 00:53:39.056 at the early time point? 00:53:39.536 --> 00:53:45.076 Or do we want a monotonic model like in High SPY 2, there was a monotonic model 00:53:45.076 --> 00:53:46.846 that the more reduction, the better. 00:53:47.406 --> 00:53:52.046 So you're learning about all of the values from neighboring ones as well potentially. 00:53:52.886 --> 00:53:58.916 That if you're a two at 30 days, do we want forecasts to be better, which 00:53:58.916 --> 00:54:00.586 is a better value, than a three? 00:54:01.516 --> 00:54:05.686 So do we want that, or do we want to only look at patients that are on that value? 00:54:05.686 --> 00:54:07.646 Might even have non-monotonic forecasts. 00:54:08.036 --> 00:54:12.036 These are all the things that go into building this adaptive design 00:54:12.036 --> 00:54:15.606 machinery that, by the way, when the trial's over, the machinery goes away 00:54:15.756 --> 00:54:20.266 unless you're using it for missing data within the circumstances of it. 00:54:20.596 --> 00:54:26.166 We've actually now done a number of stroke trials where we're using 24-hour NIHSS, 00:54:26.446 --> 00:54:33.176 NIH stroke score, as forecasting 90 days, and it's actually really good forecasting. 00:54:33.546 --> 00:54:34.936 It's really, really good. 00:54:34.976 --> 00:54:41.426 And then updating it based on seven-day status, 30-day status to 90-day status. 00:54:41.426 --> 00:54:45.066 Again, accelerating the learning to make a better trial design. 00:54:46.536 --> 00:54:49.756 Okay, number of other examples of this. 00:54:50.466 --> 00:54:57.536 Wide range of types of variables, wide range of, of, of potential outcome 00:54:57.536 --> 00:55:01.626 variables, spending a lot of time on what, what those models look like. 00:55:01.626 --> 00:55:02.716 How do we make inferences? 00:55:02.716 --> 00:55:03.866 What do we know going in? 00:55:03.866 --> 00:55:05.016 Prior distributions. 00:55:05.446 --> 00:55:12.716 All in the name of accelerating learning to make a more efficient adaptive trial, 00:55:12.806 --> 00:55:18.256 treat patients better, make better development decisions, enroll patients 00:55:18.256 --> 00:55:23.066 in a more efficient way, making a better design through this mechanism. 00:55:24.866 --> 00:55:28.916 All right, appreciate you joining me here today. 00:55:29.446 --> 00:55:32.846 Appreciate you joining me in the fight against time. 00:55:33.296 --> 00:55:35.916 Uh, sometimes we're trying to slow time. 00:55:36.216 --> 00:55:39.126 We're all, we're all trying to slow time. 00:55:39.126 --> 00:55:41.106 Here, we're trying to speed up time. 00:55:41.836 --> 00:55:47.956 We want to improve learning, speed up learning through longitudinal modeling. 00:55:49.316 --> 00:55:54.596 All right, I hope you can make your adaptive trials more efficient, and until 00:55:54.596 --> 00:55:57.736 next time, we'll be here in the interim.