WEBVTT NOTE This file was generated by Descript 00:00:00.029 --> 00:00:03.319 Judith: Welcome to Berry's In the Interim podcast, where we explore the 00:00:03.319 --> 00:00:07.389 cutting edge of innovative clinical trial design for the pharmaceutical and 00:00:07.389 --> 00:00:09.500 medical industries, and so much more. 00:00:10.229 --> 00:00:10.949 Let's dive in. 00:00:12.687 --> 00:00:13.227 Scott: All right. 00:00:13.227 --> 00:00:14.187 Welcome everybody. 00:00:14.187 --> 00:00:15.987 Back to in the interim. 00:00:16.497 --> 00:00:17.667 I'm Scott Berry. 00:00:17.667 --> 00:00:22.917 I'm your host, and I am a statistician, as, as many of you 00:00:22.917 --> 00:00:25.587 know, uh, Bayesian statistician. 00:00:25.587 --> 00:00:28.587 And here on, in the interim, we explore things. 00:00:29.027 --> 00:00:34.397 Around clinical trial science, medical decision making, uh, data 00:00:34.397 --> 00:00:37.367 analytics, uh, this is our space. 00:00:37.367 --> 00:00:39.317 We, we have a new studio here. 00:00:39.317 --> 00:00:43.097 We're we're, our technology has changed a little bit, and for those 00:00:43.097 --> 00:00:47.657 of you who consume this via video, looks like a bit of a different place. 00:00:48.537 --> 00:00:52.197 Um, I am, I am recording from a different place here. 00:00:52.707 --> 00:00:57.987 Uh, as, as you'll see throughout multiple different places, I was, 00:00:58.047 --> 00:01:02.697 I was traveling to this place as I introduced the topic for today. 00:01:03.447 --> 00:01:10.857 I was traveling here and I was, I was connecting through an airport and I had, 00:01:10.947 --> 00:01:15.087 uh, it was a bit of a long day and I, I was looking to get something to eat. 00:01:15.987 --> 00:01:22.377 And I went by a, uh, hamburger place with, with quite a line, and 00:01:22.377 --> 00:01:24.117 I'm deciding do I want to go there? 00:01:24.117 --> 00:01:25.437 Do I wanna wait in the line? 00:01:25.437 --> 00:01:27.537 And I decide, no, I, I don't want that. 00:01:27.867 --> 00:01:28.257 Choice. 00:01:28.257 --> 00:01:35.277 I I and because I believe at one of the places next door with a really short 00:01:35.277 --> 00:01:42.662 line that there's a, a really good, uh, uh, chicken bowl, uh, by the way, a, a 00:01:42.667 --> 00:01:45.207 a bit of fan of kava if you have kava. 00:01:45.207 --> 00:01:47.637 So I think that's what I'm getting. 00:01:47.637 --> 00:01:50.247 And so I say no to the burger place. 00:01:50.702 --> 00:01:54.957 I, I, and I go and I to get my chicken bowl. 00:01:56.022 --> 00:02:00.282 And so I get this, and it turns out it wasn't at all what I was expecting. 00:02:00.732 --> 00:02:03.432 And I got something very different than that. 00:02:03.762 --> 00:02:08.472 And it turned out to be almost kind of a chicken salad type thing. 00:02:08.802 --> 00:02:12.432 And it was interesting because then I got that and I'm, I'm 00:02:12.432 --> 00:02:13.782 walking by the burger place. 00:02:13.962 --> 00:02:15.732 You know, I, I, I really wish I would've. 00:02:15.752 --> 00:02:18.362 Gotten the burger, uh, in this setting. 00:02:19.052 --> 00:02:23.012 Uh, it's the, the, the, the trials and tribulations of travel. 00:02:23.042 --> 00:02:28.022 Well, it, it introduces a bit of my topic today, and the, the, the 00:02:28.022 --> 00:02:36.452 title of today's discussion is Regulators False Choices and ICHE 20. 00:02:37.052 --> 00:02:43.022 And so I'm gonna try to tie this all together in, in what I mean by this. 00:02:43.992 --> 00:02:50.052 So I'll start by talking about the ICHE 20, uh, Kurt and 00:02:50.052 --> 00:02:52.902 I did a discussion of this. 00:02:52.902 --> 00:02:59.382 We, you can go back and, uh, previous EP episode of the ICHE 20, it's 00:02:59.382 --> 00:03:04.992 a draft guidance document, uh, that discusses adaptive trials. 00:03:06.042 --> 00:03:07.782 Now, most of this is good. 00:03:08.112 --> 00:03:13.992 Most of the ICHE 20 is good in the sense that the existence of it, that 00:03:13.992 --> 00:03:17.052 adaptive designs are confirmatory trials. 00:03:17.172 --> 00:03:21.792 And by the way, backing up ICHE 20 is very much a harmonization. 00:03:22.092 --> 00:03:24.672 And so what goes in there is largely. 00:03:25.032 --> 00:03:31.482 Harmonized, essentially agreed throughout many, many regulators, global regulators. 00:03:31.482 --> 00:03:35.322 And so in some level, this is the floor of things. 00:03:35.322 --> 00:03:40.122 Now, different, different, uh, regions are gonna have different behaviors on things, 00:03:40.122 --> 00:03:42.312 but to some extent a floor level things. 00:03:42.312 --> 00:03:47.652 So from that perspective, adaptive designs are confirmatory. 00:03:48.237 --> 00:03:49.317 Sample size. 00:03:49.317 --> 00:03:51.567 Adaptations are confirmatory. 00:03:52.107 --> 00:03:58.317 Enrichment trials are confirmatory, seamless trials are confirmatory response. 00:03:58.317 --> 00:04:04.047 Adaptive allocation is confirmatory and it has a small set on 00:04:04.047 --> 00:04:06.447 Bayesian being confirmatory. 00:04:07.137 --> 00:04:08.622 All of these things are in there. 00:04:09.987 --> 00:04:15.957 Now, as you can go back to that there, part of this is, it's written from a 00:04:15.957 --> 00:04:18.717 pretty, um, pretty negative viewpoint. 00:04:18.792 --> 00:04:24.177 Uh, uh, I'll say that from the perspective of a lot of it is very cautionary. 00:04:24.657 --> 00:04:30.357 It, it's not inviting in that perspective, but yet its existence 00:04:30.357 --> 00:04:32.637 is still a positive thing. 00:04:33.327 --> 00:04:35.247 I wanna jump into. 00:04:35.587 --> 00:04:38.947 What I'll call the paragraph and. 00:04:39.342 --> 00:04:45.972 It's because this paragraph has been, uh, used, uh, uh, almost weaponized. 00:04:45.972 --> 00:04:47.502 Now I'm gonna go into this. 00:04:47.502 --> 00:04:50.952 I'm gonna talk about regulators and I, and I do wanna say this upfront. 00:04:50.952 --> 00:04:55.182 I think regulators do an amazing job and their perspective is different. 00:04:55.182 --> 00:04:56.712 I've never been a regulator. 00:04:57.012 --> 00:05:02.442 I've been proposing designs to regulators for more than 25 years, 00:05:02.802 --> 00:05:05.262 and I think they do a tremendous job. 00:05:05.322 --> 00:05:07.902 So part of this may border on what you. 00:05:07.977 --> 00:05:12.567 You consider criticism, and I'm gonna try to put myself in the mind of a 00:05:12.567 --> 00:05:17.877 regulator and think of that perspective of what happens when you're on that 00:05:17.877 --> 00:05:20.757 side and, and evaluating designs. 00:05:21.927 --> 00:05:27.477 But from that side and perspective, there's a particular paragraph and ICHE 00:05:27.477 --> 00:05:32.067 20 that I think has been weaponized and I think the paragraph is wrong. 00:05:32.217 --> 00:05:35.637 So I thought that this would make a good podcast, uh, to 00:05:35.637 --> 00:05:37.227 be a bit controversial here. 00:05:38.412 --> 00:05:48.132 So first in ICH-E20 There's a section that talks about, this is section three 00:05:48.132 --> 00:05:53.442 and it talks about the adequacy within the development program, section 3.1, 00:05:54.042 --> 00:05:57.072 and it had, this is not the paragraph yet, but it has the 00:05:57.072 --> 00:05:59.652 sentence that says A stepwise. 00:05:59.922 --> 00:06:06.792 Program with careful analysis and evaluation of completed exploratory 00:06:06.792 --> 00:06:14.322 trials helps inform the goals and design choices for subsequent confirmatory 00:06:14.322 --> 00:06:20.652 trials, and ultimately generate data necessary for regulatory decision making. 00:06:22.212 --> 00:06:28.452 Somewhat of an inno, innocuous sentence, but it lays out this notion 00:06:28.452 --> 00:06:34.812 that a stepwise program, to some extent is a bit of the gold standard. 00:06:36.462 --> 00:06:43.227 And I, it, it, that sentence has is isn't a problem, but it, it, it 00:06:43.232 --> 00:06:48.372 sort of sets it out that almost what we're doing now is kind of right. 00:06:49.077 --> 00:06:53.067 And things that are different than that are, are pushing against that. 00:06:53.937 --> 00:06:57.057 But that moves into in the same section. 00:06:57.057 --> 00:07:00.387 And this is line 1 0 1 in this draft guidance. 00:07:00.717 --> 00:07:06.837 So this is the paragraph now, and the start of this says 00:07:06.837 --> 00:07:12.807 the number and complexity of adaptations at the confirmatory 00:07:12.807 --> 00:07:15.267 stage should generally be limited. 00:07:18.462 --> 00:07:22.062 So that's the first sentence, and I'm gonna push back on that. 00:07:22.872 --> 00:07:27.372 And then it continues increasing either of them. 00:07:27.942 --> 00:07:30.972 So the number and complexity. 00:07:31.872 --> 00:07:38.322 Increasing either of them as a replacement for a sequence of multiple 00:07:38.322 --> 00:07:45.552 trials can impair the ability to answer important clinical questions 00:07:45.912 --> 00:07:53.502 and limit the opportunity to carefully reflect on prior results to design a 00:07:53.502 --> 00:07:56.472 development program most effectively. 00:07:59.502 --> 00:08:02.442 So that's, that's the paragraph. 00:08:02.442 --> 00:08:03.462 It does continue on. 00:08:03.462 --> 00:08:07.242 I think those two sentences are, are what we'll spend a good bit of, of 00:08:07.242 --> 00:08:12.282 this, this episode, talking about, it says, before planning a confirmatory 00:08:12.282 --> 00:08:17.267 trial with multiple adaptation sponsors, sponsors should discuss whether 00:08:17.317 --> 00:08:23.952 additional exploratory trials are necessary to investigate the questions 00:08:23.952 --> 00:08:26.472 addressed by the proposed adaptations. 00:08:27.837 --> 00:08:35.367 So this paragraph has, has been somewhat weaponized in a way, and I know that 00:08:35.367 --> 00:08:44.787 sounds pejorative, but you, that this idea that the number of adaptations, the number 00:08:44.787 --> 00:08:48.417 of interims and the complexity is bad. 00:08:50.757 --> 00:08:54.417 And it's a replacement for a sequence of multiple trials. 00:08:54.417 --> 00:08:58.437 Now, remember, going back to some extent that's the right answer here. 00:08:59.487 --> 00:09:04.347 So a couple things about this, and so by the way, this has been presented where 00:09:04.347 --> 00:09:07.197 designs have been presented to regulators. 00:09:08.457 --> 00:09:14.307 They've come back and they've pushed back on the design some form of 00:09:14.307 --> 00:09:17.937 no, and they've cited this draft. 00:09:18.717 --> 00:09:21.657 Paragraph for the reason for that. 00:09:22.077 --> 00:09:24.147 It's too complex. 00:09:24.837 --> 00:09:31.677 Do a separate trial and then a simple phase three confirmatory trial. 00:09:32.577 --> 00:09:37.767 So within the guidance document on adaptive trials that says 00:09:37.767 --> 00:09:43.527 all of this is confirmatory, there's a paragraph that really. 00:09:43.977 --> 00:09:48.987 Limits much of it to something that I'm gonna push back on, 00:09:48.987 --> 00:09:50.607 and I think it's largely wrong. 00:09:51.627 --> 00:09:59.487 So first, there's this, this general thought that more adaptations is 00:09:59.487 --> 00:10:07.317 somehow bad or somehow not good. 00:10:07.767 --> 00:10:10.122 I think this is largely a myth. 00:10:11.427 --> 00:10:14.607 I don't believe it creates more bias. 00:10:14.877 --> 00:10:15.087 Yeah. 00:10:15.087 --> 00:10:17.307 It's operationally more complex by the way. 00:10:17.307 --> 00:10:18.117 Marginally. 00:10:18.117 --> 00:10:21.867 It's to do, you know, seven interims compared to six is 00:10:21.867 --> 00:10:23.817 marginally very little to do. 00:10:23.817 --> 00:10:26.457 Two compared to one is marginally very little. 00:10:26.637 --> 00:10:29.547 That first one in setting up the process of how this is 00:10:29.607 --> 00:10:31.857 done is an operational lift. 00:10:32.397 --> 00:10:37.437 It's it's minor relative to the general operational burden of 00:10:37.437 --> 00:10:39.717 running a clinical trial, but. 00:10:40.512 --> 00:10:47.262 I, I think this is largely somewhat of a myth that multiple adaptations 00:10:47.262 --> 00:10:49.902 in some way increases potential bias. 00:10:51.702 --> 00:10:57.162 And to some extent, I think it almost decreases it when you do a single 00:10:57.162 --> 00:11:03.252 adaptation in a trial, a single futility or a single success analysis. 00:11:03.252 --> 00:11:07.452 And much of these, what I'll call myths, come from the traditional 00:11:07.452 --> 00:11:12.162 thing of the only adaptations or stopping for success, yes or no. 00:11:12.852 --> 00:11:16.242 And now trials have moved on greatly as many of those that I 00:11:16.242 --> 00:11:17.802 read that are in this guidance. 00:11:18.447 --> 00:11:20.667 Uh, the possibility of those. 00:11:21.087 --> 00:11:27.717 So when you do one of these analyses and you continue on, or you don't stop or you 00:11:27.717 --> 00:11:31.467 jump futility, there's something learned about what's going on in the trial. 00:11:31.677 --> 00:11:38.307 It's, you know, depending on when it, it, the relatively minor, but people 00:11:38.307 --> 00:11:42.417 can perceive there to be potential biases of that when you do one. 00:11:42.657 --> 00:11:44.787 That's the biggest potential there. 00:11:45.537 --> 00:11:50.127 If you did a hundred in a trial, each one of these, I, I mean, 00:11:50.127 --> 00:11:51.507 it's just, it's like daily. 00:11:51.507 --> 00:11:56.787 These are being done essentially ignored, and we've done trials. 00:11:56.787 --> 00:11:59.877 I SPY two used to do interims daily. 00:12:00.537 --> 00:12:02.817 It, it eventually went to weekly. 00:12:03.177 --> 00:12:06.447 And I think by the way, trials are more likely to go that way. 00:12:06.852 --> 00:12:11.022 It essentially removes these, these types of concerns. 00:12:11.442 --> 00:12:16.602 So I, I wanna push back on that, but that's not even necessarily 00:12:16.602 --> 00:12:18.972 the, the, the large extent of it. 00:12:19.812 --> 00:12:25.002 It's that overall this sets up, and this is where I'm gonna get into 00:12:25.062 --> 00:12:27.792 this idea of sort of false choices. 00:12:28.422 --> 00:12:34.422 This somewhat sets up the notion that adaptations are shortcuts. 00:12:36.837 --> 00:12:44.967 That when the alternative is doing multiple exploratory trials, that 00:12:44.967 --> 00:12:47.667 somehow adaptations are always shortcuts. 00:12:47.697 --> 00:12:52.047 And by the way, when you read this ICHE 20, that's an overriding 00:12:52.047 --> 00:12:54.447 thought about this guidance document. 00:12:54.447 --> 00:12:59.487 It's, it's the part that when you read it, much of it is set up in that way. 00:13:00.777 --> 00:13:05.577 And I think it's a false choice, and I think false choices set that up. 00:13:06.027 --> 00:13:08.277 And so I wanna walk through this a little bit. 00:13:08.277 --> 00:13:12.807 I'll walk through it with examples and try to explain what 00:13:12.807 --> 00:13:15.567 I, what I mean by that from it. 00:13:17.667 --> 00:13:20.637 So I, I, I'll start with an example trial. 00:13:20.637 --> 00:13:24.957 And this trial, uh, was conducted several years ago. 00:13:24.957 --> 00:13:30.687 The trial's over the name of the trial is Sepsis Act, and it was a 00:13:30.687 --> 00:13:33.327 trial run by faring pharmaceuticals. 00:13:33.597 --> 00:13:39.087 It was the treatment for sepsis and they had a treatment, uh, a 00:13:39.087 --> 00:13:45.327 form of a long-acting vasopressor, and they had three doses. 00:13:46.302 --> 00:13:48.012 And to explore. 00:13:49.332 --> 00:13:53.952 And they were, they wanted to explore the, the effective dose and they wanted 00:13:53.952 --> 00:13:58.602 to explore the effective dose on clinical outcomes, not necessarily biomarkers. 00:13:58.932 --> 00:14:06.342 So the role is a long-acting vasopressor going to be beneficial on clinical 00:14:06.342 --> 00:14:08.562 outcomes for patients with sepsis. 00:14:10.182 --> 00:14:13.182 So they have three doses in placebo, and the question is, 00:14:13.182 --> 00:14:14.697 what kind of trial do they run? 00:14:15.867 --> 00:14:19.677 And they want to explore dose, and then they want to run a phase three trial. 00:14:20.547 --> 00:14:23.817 Well, when we started looking at this, initially, they thought of, 00:14:23.817 --> 00:14:28.077 okay, maybe we do a phase two trial, separate sequential trials, and 00:14:28.077 --> 00:14:30.537 explore three doses in placebo. 00:14:30.927 --> 00:14:33.207 Well, to get some idea of the effect. 00:14:33.212 --> 00:14:37.922 Of dose on clinical outcomes like length of, of organ support and 00:14:37.922 --> 00:14:42.242 mortality, that the endpoint in this trial ends up being a composite of 00:14:42.242 --> 00:14:45.602 mortality and length of organ support. 00:14:46.172 --> 00:14:52.172 For those that survive an ordinal endpoint, that 200 patients, or a 00:14:52.172 --> 00:14:56.042 hundred patients, or 200 patients was, was insufficient to select dose. 00:14:56.042 --> 00:14:56.672 It's, it's too small. 00:14:58.182 --> 00:15:03.792 As you start to get into 400, 500, 600 patient trials, you're almost starting 00:15:03.792 --> 00:15:05.772 to get into phase three territory. 00:15:06.312 --> 00:15:11.232 So to run a realistic phase two trial to understand clinical outcomes 00:15:11.232 --> 00:15:14.022 starts to get to 5, 6, 700 patients. 00:15:14.472 --> 00:15:17.382 Well, you now you're getting into mini phase three trials. 00:15:18.642 --> 00:15:22.332 So they explored this and they ended up with, here's, uh, I'll 00:15:22.332 --> 00:15:24.282 describe the Sepsis ACT trial. 00:15:25.302 --> 00:15:29.652 Where, when a hundred patients are enrolled equal randomization 00:15:29.652 --> 00:15:35.592 to three doses and, and placebo response, adaptive randomization 00:15:35.592 --> 00:15:38.202 starts only on the active doses. 00:15:38.202 --> 00:15:41.502 The, the randomization to control is fixed. 00:15:42.912 --> 00:15:47.772 And by the way, it's fixed at, uh, a third, so 33% and then 00:15:47.772 --> 00:15:50.412 the remaining 67 are set up. 00:15:50.412 --> 00:15:56.232 So it wasn't equally to all of them, but the doses are equal and placebo's 33%. 00:15:56.712 --> 00:16:01.722 The response adaptive over these three doses, it also had the possibility 00:16:01.722 --> 00:16:05.502 of adding a fourth dose if the dose response looked increasing. 00:16:06.132 --> 00:16:06.852 Within that. 00:16:07.122 --> 00:16:12.672 So a neat little design that at a hundred, these RAR updates start. 00:16:13.962 --> 00:16:18.222 Once 200 patients are enrolled, and by the way, those happen monthly. 00:16:18.882 --> 00:16:22.212 So every month RAR is reset. 00:16:22.362 --> 00:16:28.062 So now we're starting to do a good number of interims In this stage. 00:16:28.062 --> 00:16:35.007 One of the trial, once 200 patients are enrolled, the trial could go to stage two. 00:16:36.027 --> 00:16:42.567 That would be selecting a single dose and moving to one-to-one randomization. 00:16:43.017 --> 00:16:48.987 And the final sample size would be 1800 patients, including stage one, 00:16:50.637 --> 00:16:56.577 and that would form a phase three trial and interims keep happening 00:16:56.577 --> 00:17:02.067 monthly, and it could go from 200, anywhere up to 800 could be the 00:17:02.067 --> 00:17:04.377 dose ranging portion of this trial. 00:17:05.427 --> 00:17:08.847 At 800, there's a forced decision. 00:17:09.627 --> 00:17:13.527 Now you have to go to stage two or you have to stop, and it's a 00:17:13.527 --> 00:17:17.307 large phase two trial, but largely you'd be stopping for no go. 00:17:17.922 --> 00:17:21.657 I, I won't call it futility, but it's a no Go to phase three, and 00:17:21.657 --> 00:17:23.577 it becomes a large phase two trial. 00:17:25.182 --> 00:17:28.662 If things are going very well and you've identified a dose, it could go 00:17:28.662 --> 00:17:34.932 earlier and it's eight 1800 is the, is the max, is the final sample size, and 00:17:34.932 --> 00:17:36.912 so there's no adaptive sample size. 00:17:36.912 --> 00:17:43.182 If this goes to phase three, the trial ended up running, uh, more than 20 00:17:43.182 --> 00:17:45.312 interim analysis during the course of it. 00:17:45.702 --> 00:17:54.042 And this, this is considered a complex design, multiple interims and. 00:17:55.107 --> 00:18:03.087 So now I, I'll, I'll claim that if this, this design would be submitted 00:18:03.237 --> 00:18:09.027 in the presence of that paragraph, it would be very easy for a regulator to 00:18:09.027 --> 00:18:15.597 look at that design and say, that's too complex, too many interims. 00:18:16.077 --> 00:18:23.457 Go run a separate phase two sequential trials are better as set up. 00:18:24.402 --> 00:18:27.912 Go run an 1800 patient phase two trial. 00:18:28.782 --> 00:18:32.442 Then when you've selected dose, made your decisions go run an 00:18:32.442 --> 00:18:34.842 1800 patient phase three trial. 00:18:35.982 --> 00:18:40.722 So as a regulator, maybe that's what you're evaluating. 00:18:40.782 --> 00:18:45.792 So when presented this seamless two three trial with a, with 00:18:45.822 --> 00:18:47.077 a good number of interims. 00:18:48.042 --> 00:18:50.922 In the trial, a fair amount of complexity. 00:18:51.312 --> 00:18:54.822 And again, I'll back up and think that these, these interims 00:18:54.822 --> 00:18:56.292 aren't causing more bias. 00:18:56.292 --> 00:18:59.112 They're not somehow risking this. 00:18:59.112 --> 00:19:01.122 It's not a bad thing that these are done. 00:19:01.122 --> 00:19:03.882 I think in some ways they de-risk all of it, but 00:19:06.012 --> 00:19:12.522 the the when, when saying, oh no, you can't do that trial. 00:19:13.842 --> 00:19:16.152 They think the alternative. 00:19:17.232 --> 00:19:23.082 Is an 800 patient phase two, followed by an 1800 patient, phase three. 00:19:25.032 --> 00:19:28.752 The problem is, that's not the alternative, that's a false choice 00:19:29.742 --> 00:19:34.272 as this extreme sort of value is, oh no, I don't want that. 00:19:34.452 --> 00:19:38.142 I, I'm gonna get this other thing, but you're not getting that. 00:19:38.652 --> 00:19:44.682 So you say no to design a and now you've got a paragraph that somewhat weaponizes. 00:19:45.222 --> 00:19:47.172 A reason to say no to that. 00:19:47.202 --> 00:19:53.502 Hoping you're going to get design B 800 plus 1800, but you 00:19:53.502 --> 00:19:56.382 don't get B, you get design C. 00:19:57.462 --> 00:20:00.012 What do you get in that circumstance? 00:20:00.672 --> 00:20:05.502 Well, you either get that they don't explore dose and they go to phase three 00:20:05.772 --> 00:20:13.272 or a small 200 patient trial that actually does a really poor job at dose finding. 00:20:13.272 --> 00:20:14.142 Go no go. 00:20:14.772 --> 00:20:17.742 Followed by 1800 in that. 00:20:19.662 --> 00:20:23.802 So the design you want, which is 2,600 patients in the 00:20:23.802 --> 00:20:26.082 separate, is never gonna be run. 00:20:26.832 --> 00:20:28.092 It's a false choice. 00:20:28.092 --> 00:20:33.942 And the reason for that, now coming back to this idea within this trial 00:20:34.332 --> 00:20:37.632 is, is this adaptive design a shortcut? 00:20:39.372 --> 00:20:41.562 And it is a shortcut. 00:20:42.267 --> 00:20:50.817 When evaluated compared to 800 and then 1800, it's a shortcut. 00:20:51.747 --> 00:20:52.587 Uh, in that setting. 00:20:52.587 --> 00:20:55.227 It's combining the 800 with the 1800. 00:20:55.587 --> 00:20:57.927 Now we can control type 1 error. 00:20:57.927 --> 00:21:01.137 We can understand all of the characteristics of this. 00:21:01.197 --> 00:21:05.847 We can make it a confirmatory trial, but it is using 1800 00:21:05.847 --> 00:21:07.797 patients rather than 2,600. 00:21:07.797 --> 00:21:10.017 It's efficient within that perspective. 00:21:10.872 --> 00:21:16.002 So, but compared to 800 and 1800, you could claim it's a shortcut, but that's 00:21:16.002 --> 00:21:18.012 a design that's never going to be run. 00:21:19.692 --> 00:21:26.502 What would be run is 200 plus 1800, and the 1800 with the seamless 00:21:26.502 --> 00:21:28.752 up to 800 is a better design. 00:21:29.322 --> 00:21:31.812 It's better design for regulators. 00:21:31.812 --> 00:21:34.332 It's a better design for patients. 00:21:34.572 --> 00:21:36.972 It's a better design for sponsors. 00:21:37.512 --> 00:21:39.672 Everything about it is better. 00:21:42.372 --> 00:21:46.362 And that's the value of adaptive designs here. 00:21:46.602 --> 00:21:52.152 And efficiency isn't shortcut efficiency is the ability to answer questions that 00:21:52.152 --> 00:21:54.192 aren't going to be done separately. 00:21:56.532 --> 00:21:56.802 Now. 00:21:58.437 --> 00:22:05.577 Go back to episode 17, by the way, where we, we, I, I talk about a paper, uh, with 00:22:05.577 --> 00:22:08.367 Lindsey Berry, Joe Mary, and Kurt Veley. 00:22:08.787 --> 00:22:16.347 If you have 1800 patients to use as a sponsor and you have three doses to 00:22:16.347 --> 00:22:23.727 explore, a seamless two three trial is going to dominate in terms of getting 00:22:23.727 --> 00:22:26.187 the right dose in terms of power. 00:22:27.072 --> 00:22:33.222 Any other sequential use of those 1800 patients, you could break 00:22:33.222 --> 00:22:39.402 it up into 500 and 1300, uh, compared to a seamless trial. 00:22:39.402 --> 00:22:46.272 And that paper shows, uh, uh, Barry et all shows that that's the optimal 00:22:46.272 --> 00:22:50.202 way, improves power, dose selection, everything about it is better. 00:22:52.092 --> 00:22:54.342 Better than separate trials. 00:22:55.122 --> 00:23:00.317 So that's the part of this that I think's a false choice is. 00:23:01.977 --> 00:23:03.897 No, I don't want A, I want B. 00:23:03.897 --> 00:23:05.577 But you don't get B, you get C. 00:23:06.777 --> 00:23:08.847 It happened to me traveling yesterday. 00:23:09.057 --> 00:23:13.437 I said no to a thinking I was gonna get, you know, I said no to the hamburger 00:23:13.797 --> 00:23:15.597 thinking I was getting something else. 00:23:15.597 --> 00:23:17.487 What I got was not that. 00:23:17.877 --> 00:23:20.787 Uh, I would've gotten the hamburger had I had the choice. 00:23:20.847 --> 00:23:25.137 The same thing from the regulator perspective here now. 00:23:26.367 --> 00:23:27.207 What does this mean? 00:23:27.207 --> 00:23:32.667 Interestingly, by the way that trial design was, was run, it was given a spa, 00:23:32.727 --> 00:23:35.202 a special protocol assessment by the FDA. 00:23:36.267 --> 00:23:38.487 Was approved, they said yes. 00:23:38.787 --> 00:23:43.107 Uh, there were, there were various interactions and I think the interactions 00:23:43.107 --> 00:23:45.507 really were in what, what would be done. 00:23:45.507 --> 00:23:51.297 And it was presented, and by the way, this did go to multiple regulators as. 00:23:51.597 --> 00:23:56.457 Why don't you do a 200 patient phase two trial and then a phase three trial and 00:23:56.457 --> 00:24:01.257 it, we were able to present the different operating characteristics of this, and I 00:24:01.257 --> 00:24:06.087 think it was with the ability to see the range of potential possibilities here. 00:24:06.237 --> 00:24:10.167 This is actually a pretty good design for all parties involved, 00:24:10.167 --> 00:24:11.997 and it's not a shortcut. 00:24:13.197 --> 00:24:16.527 Adaptive designs are not shortcuts. 00:24:17.457 --> 00:24:20.217 Efficiency is not shortcuts. 00:24:21.267 --> 00:24:25.617 Depending on what you think the possibility of those designs are. 00:24:26.487 --> 00:24:27.027 Okay. 00:24:27.597 --> 00:24:31.497 Interestingly, in that trial where these interims were run, I was actually 00:24:31.497 --> 00:24:33.897 unblinded during the course of this trial. 00:24:34.317 --> 00:24:39.507 Had this trial been a 200 patient, phase two, they would've run phase three. 00:24:40.077 --> 00:24:43.977 They would've run an 1800 patient phase, uh, phase three trial. 00:24:44.457 --> 00:24:46.827 The data at that time were quite positive. 00:24:46.917 --> 00:24:49.527 Now, by the time 800 rolled around. 00:24:50.427 --> 00:24:53.577 It wasn't positive and it stopped for no-go. 00:24:54.657 --> 00:24:59.007 And you can go, you can look up the various characteristics of, uh, uh, 00:24:59.007 --> 00:25:02.997 the results have been reported on this, and the trial was a no-go. 00:25:02.997 --> 00:25:04.227 It didn't go to phase three. 00:25:04.947 --> 00:25:09.507 It ended up using approximately 800 patients, uh, with, and I think the, 00:25:09.507 --> 00:25:10.917 the result of it was quite good. 00:25:10.917 --> 00:25:11.877 200 patients. 00:25:11.877 --> 00:25:15.597 It would've conducted a very large, expensive phase three trial with 00:25:15.597 --> 00:25:20.367 CMC and the expansion to it, which I think would've been the wrong choice 00:25:20.697 --> 00:25:25.917 now, uh, that, that's, that's one particular outcome of the trial, but 00:25:25.917 --> 00:25:29.187 we simulated all the possibilities and we know the probability 00:25:29.187 --> 00:25:30.837 that of these things happening. 00:25:31.482 --> 00:25:34.632 From a 200 patient trial is quite different from the 800 and that was 00:25:34.632 --> 00:25:37.332 part of how this trial was built. 00:25:40.152 --> 00:25:47.382 So this, this idea is a lot of times a design is presented to regulators and 00:25:47.382 --> 00:25:49.932 they're evaluating that single design. 00:25:51.372 --> 00:25:55.482 And so when you're evaluating that, it's always well compared to what, 00:25:55.692 --> 00:26:01.122 and that's the, that's what as, as somebody who for 25 plus years have been 00:26:01.122 --> 00:26:05.532 submitting design to regulators, a lot of times we submit that single design. 00:26:06.342 --> 00:26:09.762 And yes, there's a line I use quite frequently, I'm sure I've used it on 00:26:09.762 --> 00:26:14.232 another podcast, is when somebody asks me as a statistician, Hey, how's your wife? 00:26:14.652 --> 00:26:16.752 Uh, the answer's well compared to what? 00:26:18.012 --> 00:26:22.182 Now my wife, by the way, listens to all these podcasts, uh, and 00:26:22.182 --> 00:26:26.502 she's a saint, as you can imagine, uh, putting up with, with this. 00:26:26.862 --> 00:26:29.472 Uh, but that's the same sort of thing. 00:26:29.472 --> 00:26:32.022 It's what compared to what and if you're. 00:26:32.412 --> 00:26:39.072 In the mindset of, okay, I've been presented this design a boy. 00:26:39.072 --> 00:26:45.072 I'd rather have them do separate trials, big trials sort of thing. 00:26:45.222 --> 00:26:49.602 If, if, if I'm thinking that's my comparison, oh, I, I, I might say no to 00:26:49.602 --> 00:26:56.472 A, because I think I'm getting B, but B is not possible and it's not gonna be run. 00:26:56.472 --> 00:26:57.972 And you get c. 00:26:59.397 --> 00:27:01.377 So a recent example of this. 00:27:01.377 --> 00:27:04.827 So part of this is how do we, how do we help this process? 00:27:05.127 --> 00:27:08.517 And if, if I were a regulator, I might very well do the same thing. 00:27:08.517 --> 00:27:11.007 So I don't want this to be a criticism of regulators. 00:27:11.007 --> 00:27:13.677 They sit in a different chair and they're evaluating a 00:27:13.677 --> 00:27:16.377 single design, a single thing. 00:27:16.377 --> 00:27:17.517 How, how is that? 00:27:17.517 --> 00:27:21.717 Well, I mean, compared to what, and I might think, oh, you should do, you should 00:27:21.717 --> 00:27:23.577 do this adaptive design sort of thing. 00:27:23.787 --> 00:27:25.617 So I, I push back on a. 00:27:25.872 --> 00:27:28.482 But I don't get a better adaptive design. 00:27:28.542 --> 00:27:31.332 Maybe I get a design I think is worse than that one. 00:27:31.632 --> 00:27:34.122 So that's a challenge in this interaction. 00:27:34.122 --> 00:27:37.992 You usually throw a single design and say, Hey, how about this design? 00:27:38.592 --> 00:27:42.162 A recent example of this was another seamless two three trial. 00:27:42.882 --> 00:27:49.062 It explored two doses and, uh, a relatively rare disease, by the way, 00:27:49.332 --> 00:27:55.962 exploring two doses and picking a dose, moving to one-to-one randomization, 00:27:56.232 --> 00:28:01.482 and going to a section that is then, uh, uh, adaptive sample size. 00:28:01.482 --> 00:28:06.402 So three interims to select sample size once it moves to one-to-one, 00:28:06.942 --> 00:28:11.472 and that that's the trial design and submitted to regulators. 00:28:13.092 --> 00:28:15.372 Now there's some pushback on that. 00:28:15.492 --> 00:28:21.342 And again, that paragraph could be used to say, aha, you're doing dose 00:28:21.342 --> 00:28:24.432 selection, you're doing several interims. 00:28:24.672 --> 00:28:30.192 That's too complex to be confirmatory. 00:28:30.612 --> 00:28:34.002 Go run a separate phase two and then run phase three. 00:28:34.182 --> 00:28:40.542 And this was actually the advice given by regulars given by the FDA within that. 00:28:42.447 --> 00:28:45.927 When this comes back and you're working with the sponsor and you 00:28:45.927 --> 00:28:47.967 say, okay, how about the possibility? 00:28:48.087 --> 00:28:51.957 Oh, and, and by the way, part of the reason was the pushback is 00:28:51.957 --> 00:28:55.497 the endpoint in this trial is a relatively long-term endpoint. 00:28:57.012 --> 00:29:02.442 And so we're making dose selection based on early predictive markers. 00:29:02.622 --> 00:29:07.092 Now, those markers are not the confirmatory marker endpoint at the 00:29:07.092 --> 00:29:09.252 end, which is the longer term agreed. 00:29:09.612 --> 00:29:12.252 Uh uh, fields function survives endpoint. 00:29:13.992 --> 00:29:19.212 But we have to, because of enrollment and 'cause of the scenario base it, there will 00:29:19.212 --> 00:29:23.322 be a little bit of long-term endpoint, but largely it's an earlier endpoint and the 00:29:23.322 --> 00:29:28.362 FDA was concerned about the predictability of that and they would rather have a 00:29:28.362 --> 00:29:31.932 separate phase two that goes all the way out, long-term clinical outcome, 00:29:31.962 --> 00:29:33.732 make your decision and run phase three. 00:29:33.732 --> 00:29:35.052 And that was the advice given. 00:29:35.562 --> 00:29:38.472 So now the regulator gets this design A and thinks, oh, 00:29:38.472 --> 00:29:39.852 that would be a better design. 00:29:40.562 --> 00:29:47.192 And you can always create a superset design to an adaptive design that is 00:29:47.192 --> 00:29:58.052 better, but that would then be this larger, longer phase 2 larger phase 3 00:29:59.457 --> 00:30:04.497 And when going back to the sponsor and saying, okay, is that a possibility? 00:30:04.527 --> 00:30:05.727 It's impossible. 00:30:06.447 --> 00:30:08.157 They said the company would fold. 00:30:08.157 --> 00:30:09.807 That trial can't be run. 00:30:11.577 --> 00:30:18.567 So that, I mean, that's where this falls into it is any adaptive design. 00:30:18.567 --> 00:30:22.497 You can create a design that's a superset of it and say, oh, you should do that. 00:30:22.647 --> 00:30:24.087 It's not a possible choice. 00:30:24.117 --> 00:30:25.707 You, you, that you can't do that. 00:30:25.707 --> 00:30:29.217 So saying no to A, because I want B, it's a false choice. 00:30:31.692 --> 00:30:35.577 So what the, what, what's if somebody, if regulator says, no, 00:30:35.577 --> 00:30:38.847 you can't do design a what gets run. 00:30:39.477 --> 00:30:44.607 Is a phase 3 trial without dose selection, and that's what the sponsor said. 00:30:44.697 --> 00:30:49.167 If we can't run that trial that selects a dose based on earlier interval, we're 00:30:49.167 --> 00:30:55.287 just gonna pick a dose and run phase 3 What you get back is not why you said 00:30:55.287 --> 00:31:00.357 no to a, so all of a sudden, suppose the regulators understood that the 00:31:00.357 --> 00:31:06.987 choice was that design A, that at least selects dose based on an earlier marker. 00:31:09.027 --> 00:31:13.467 And then moves into the seamless phase 2/3 part of it into the confirmatory 00:31:13.467 --> 00:31:17.697 part, or you just get a phase 3 and they pick the dose with no data. 00:31:18.927 --> 00:31:25.077 I think they'd pick design A, and they'd live with picking it on that endpoint, but 00:31:25.077 --> 00:31:28.347 when evaluating it, in their mind they're thinking, oh, I, I think you should 00:31:28.347 --> 00:31:30.507 do design B, but that's not a choice. 00:31:31.707 --> 00:31:36.117 Now, this is a bit of a challenge for us is. 00:31:36.462 --> 00:31:37.842 How do, how do we work with that? 00:31:37.842 --> 00:31:41.532 How do we present this in a way that's consumable and all of that? 00:31:41.532 --> 00:31:43.302 And I'll, I'll come back to that a little bit. 00:31:44.292 --> 00:31:48.612 What, what happened here, by the way, is a great story is in the 00:31:48.612 --> 00:31:54.252 in-person meeting, the sponsor shared that we can't run that design. 00:31:54.252 --> 00:31:55.602 We can't run separate troughs. 00:31:55.662 --> 00:31:57.222 I mean, it would not be run. 00:31:57.942 --> 00:32:02.112 The agency became a partner in this. 00:32:02.832 --> 00:32:06.882 Sort of development as where do we go? 00:32:06.942 --> 00:32:09.282 What can we do under this setting? 00:32:09.492 --> 00:32:15.402 And largely, uh, a form of a slightly a, a adapted in a good way. 00:32:16.122 --> 00:32:17.922 It was what came out of that. 00:32:18.432 --> 00:32:21.102 So it was a beautiful interaction. 00:32:21.102 --> 00:32:25.722 And again, I think the FDA did a tremendous job in being partners, in 00:32:26.022 --> 00:32:29.442 creating better care for patients. 00:32:29.772 --> 00:32:36.102 And it was, it was great, but it's this hard aspect of, of evaluating 00:32:36.102 --> 00:32:37.782 a single design in isolation. 00:32:39.807 --> 00:32:43.347 Another example of this, and I'll, I'll do this relatively quickly, was an 00:32:43.347 --> 00:32:47.967 enrichment trial where it started off one-to-one randomization phase three 00:32:47.967 --> 00:32:52.107 trial, but there's a concern about what's the right patient population 00:32:52.107 --> 00:32:55.797 and there's some risk that we might be enrolling patients who who don't benefit 00:32:56.007 --> 00:32:57.207 the, you know, there's always this. 00:32:57.927 --> 00:33:02.667 Enrolling people on the healthy side or the less healthy side maybe, that you 00:33:02.667 --> 00:33:08.937 can't really mod, uh, you, you can't modify the effect in those patients 00:33:08.937 --> 00:33:11.337 sealing and, and, and floor effects. 00:33:11.877 --> 00:33:12.687 So. 00:33:13.062 --> 00:33:16.602 Starts off enrolling a rather larger set, but could restrict 00:33:16.602 --> 00:33:21.162 enrollment, do enrichment in the trial, and to do enrichment. 00:33:21.162 --> 00:33:23.262 Well, by the way, when you simulate these trials, you 00:33:23.262 --> 00:33:24.702 have to do it relatively early. 00:33:24.702 --> 00:33:28.542 If you wait too long to do enrichment, you know your trial's largely 00:33:28.542 --> 00:33:30.312 enrolled and it has little benefit. 00:33:30.642 --> 00:33:33.462 In it, but it, so you need to do it relatively early. 00:33:33.462 --> 00:33:38.412 So the trial had three interims for potential enrichment, and then 00:33:38.412 --> 00:33:40.272 had three interims for sample size. 00:33:40.302 --> 00:33:45.162 So all of a sudden, this is a six interim phase three trial. 00:33:47.322 --> 00:33:51.462 This literally referenced the paragraph that I gave you 00:33:51.462 --> 00:33:53.622 as, this is too complicated. 00:33:54.612 --> 00:33:57.672 Doomed it by saying, this is this trial. 00:33:57.672 --> 00:34:01.932 It makes because of the complexity and the adaptations, it makes that trial 00:34:01.962 --> 00:34:05.532 exploratory, and that's, that's death. 00:34:05.532 --> 00:34:07.602 It means it's not a confirmatory trial. 00:34:08.382 --> 00:34:12.522 By the way, A different regulator region said, yes, the design's good. 00:34:12.732 --> 00:34:18.552 We like it, which is, which is a challenge here, but I think the 00:34:18.552 --> 00:34:23.442 regulator that said, no, this is exploratory, and use that paragraph. 00:34:24.882 --> 00:34:29.772 As a weapon, thought they'd get a different design, run that 00:34:29.952 --> 00:34:35.922 trial to understand the patient population and then run phase three. 00:34:37.032 --> 00:34:39.132 But that trial's never going to happen. 00:34:40.002 --> 00:34:46.242 Uh, it's unaffordable the time the, those large phase twos don't happen in that. 00:34:47.397 --> 00:34:51.447 What would happen is they guess at the patient population and they don't 00:34:51.447 --> 00:34:57.027 actually explore that and they run a phase three at risk, or they ignore 00:34:57.027 --> 00:35:00.987 the region that said no to the trial, which neither one of those is a better 00:35:00.987 --> 00:35:04.527 option than trial A of what they got. 00:35:04.527 --> 00:35:10.017 And so that's a huge aspect here and, and such a challenge, uh, in these 00:35:10.017 --> 00:35:13.467 circumstances of being a regulator because maybe you don't know what 00:35:13.467 --> 00:35:15.807 the alternatives are if you reject. 00:35:16.782 --> 00:35:17.322 The null. 00:35:17.322 --> 00:35:18.432 What's the alternative? 00:35:18.432 --> 00:35:21.282 If you reject a design, what, what, what am I gonna get? 00:35:21.672 --> 00:35:25.242 Uh, and maybe I would go back and actually really like the one 00:35:25.242 --> 00:35:26.652 that, that was already submitted. 00:35:28.632 --> 00:35:32.682 And so several parts of this, again, that design this, you know, call it. 00:35:33.207 --> 00:35:37.617 A thousand patients in this enrichment phase three trial. 00:35:37.917 --> 00:35:43.377 What they'd rather have is a thousand patient exploratory trial to find the 00:35:43.377 --> 00:35:46.707 enrichment or the enrichments only can happen up through 400, run a separate 00:35:46.707 --> 00:35:51.657 400, and then run a thousand patient phase three, but that's never gonna be run. 00:35:51.657 --> 00:35:58.767 That trial is more efficient, sorry, not more efficient, but more powerful than 00:35:58.797 --> 00:36:01.047 this seamless thousand patient trial. 00:36:02.442 --> 00:36:03.942 But that's not a choice. 00:36:04.902 --> 00:36:10.902 And so this now does better than all the alternatives and the adaptations allow 00:36:10.902 --> 00:36:12.557 a better trial to be run for everybody. 00:36:14.667 --> 00:36:16.407 Another very, very simple example. 00:36:16.407 --> 00:36:17.457 It's a very common example. 00:36:17.457 --> 00:36:20.307 This isn't a real one, but it's reflective. 00:36:20.307 --> 00:36:24.777 The other ones were real examples is reflective of what happens quite a bit. 00:36:24.777 --> 00:36:30.087 Is uh, a sponsor's interested in running a phase three trial, one-to-one randomized. 00:36:30.857 --> 00:36:34.217 Uh, double blind, all the nice things. 00:36:34.547 --> 00:36:39.707 And they say, okay, for, for, uh, an effect that they think their drug has 00:36:39.707 --> 00:36:46.247 of Delta, they get an 80, 90, it doesn't matter percent power trials, 180 patients. 00:36:47.597 --> 00:36:52.787 And then you show them, well, you know, a smaller effect than Delta, 00:36:53.237 --> 00:36:55.787 which is still clinically meaningful. 00:36:55.997 --> 00:36:59.272 Might only have 50 or 60% power with your 180. 00:37:01.257 --> 00:37:05.937 But they don't wanna run a 300 patient trial because they think 00:37:05.937 --> 00:37:09.657 they're better than that and they think that's a waste of resources. 00:37:10.197 --> 00:37:15.297 So if given the choice of a single fixed trial, they're 00:37:15.297 --> 00:37:17.097 gonna run 180 patient trial. 00:37:18.357 --> 00:37:22.617 Now the risk is that there's a clinically meaningful drug that 00:37:22.617 --> 00:37:23.847 doesn't hit success with that. 00:37:24.807 --> 00:37:29.247 So you show them what if we do several adaptations for sample size? 00:37:29.517 --> 00:37:33.117 And suppose we do interims at 150, so if the drug's even more 00:37:33.117 --> 00:37:35.187 effective, we get out sooner. 00:37:35.667 --> 00:37:40.887 200, 2 50, or 300 with appropriate futility rules. 00:37:41.802 --> 00:37:46.992 And success looks and all of a sudden this design performs really, really well 00:37:46.992 --> 00:37:52.062 because if their Delta's true, they still get out with something that on average is 00:37:52.062 --> 00:37:58.002 180 or, or better than that, they stop at 150, 200 with high probability, and that's 00:37:58.002 --> 00:38:00.552 really good if it's clinically meaningful. 00:38:00.552 --> 00:38:03.762 But less than that Delta, they still have a good chance of 00:38:03.762 --> 00:38:06.072 successor 200, 2 50, or 300. 00:38:07.227 --> 00:38:08.817 So a really nice design. 00:38:08.817 --> 00:38:13.827 Now, if somebody says, that's too complex, too many interims, they 00:38:13.827 --> 00:38:17.067 might think that because they're getting a 300 patient trial. 00:38:17.997 --> 00:38:22.047 I say no to that because run the 300 patient trial, 00:38:22.227 --> 00:38:23.937 but the 300 doesn't get run. 00:38:23.937 --> 00:38:25.912 The 180 patient trial gets run. 00:38:27.477 --> 00:38:29.247 So that's the challenge of this. 00:38:29.247 --> 00:38:32.457 This is the, the, the challenge of a regulator. 00:38:32.457 --> 00:38:36.117 And this is, this is, um, this is not necessarily their fault. 00:38:37.332 --> 00:38:42.132 In thinking about what this alternative space, and if I were a regulator, I 00:38:42.132 --> 00:38:45.912 might have a different alternative set of designs that I think could be run. 00:38:46.212 --> 00:38:49.032 And so I say, oh, you should do the following, but the those are 00:38:49.032 --> 00:38:50.622 never gonna be run potentially. 00:38:50.772 --> 00:38:55.932 And I get something back as a reaction to this that's worse than the original one. 00:38:56.232 --> 00:38:58.702 And these are the challenges of the way these are. 00:38:58.897 --> 00:39:00.277 This is particularly done. 00:39:00.847 --> 00:39:03.877 So I think we need to communicate that better, which is hard, by the 00:39:03.877 --> 00:39:09.607 way, because you can't say, if you don't take a, we're gonna do C in a 00:39:09.607 --> 00:39:12.757 way that is almost, uh, blackmail ish. 00:39:12.757 --> 00:39:16.897 That, you know, boy, if you don't say yes to our design, you're gonna get this. 00:39:17.137 --> 00:39:21.067 Uh, you know, and that, that's really not the way we traditionally 00:39:21.067 --> 00:39:23.197 when we submit a design like that. 00:39:23.857 --> 00:39:28.327 We work really hard to explain that single design as to what 00:39:28.327 --> 00:39:29.542 you're gonna get as a regulator. 00:39:30.852 --> 00:39:35.262 Now we don't spend a lot of time comparing to a bunch of other designs. 00:39:35.442 --> 00:39:39.882 It's actually can, can be confusing to somebody receiving this. 00:39:39.882 --> 00:39:41.502 What, what's the design again? 00:39:42.072 --> 00:39:46.932 But I think we need to do a better job of that, to understand the breadth 00:39:46.932 --> 00:39:52.872 of the designs that could be run, not in an aggressive way, and showing 00:39:52.872 --> 00:39:57.972 that this is a, a better design, a seamless two, three trial designs. 00:39:58.377 --> 00:40:02.217 Better design than a lot of the alternatives that could 00:40:02.217 --> 00:40:04.287 be run in that scenario. 00:40:04.827 --> 00:40:11.157 And so thi this is really our challenge as clinical trialists to be able to 00:40:11.157 --> 00:40:16.977 do this in this weird interaction we have with regulators tied to it is. 00:40:18.047 --> 00:40:23.687 As this draft guidance on ICHE 20 is going, hoping a bit of that paragraph can 00:40:23.687 --> 00:40:31.532 be massaged and less of a sharp weapon for moving forward, uh, in clinical trials. 00:40:33.897 --> 00:40:34.737 All right. 00:40:35.217 --> 00:40:41.217 I appreciate you joining me today in my, my new digs, and yes, I survive 00:40:41.247 --> 00:40:47.307 the travel and in all of that, and so I appreciate you joining us. 00:40:47.697 --> 00:40:51.567 Until next time, we'll be here in the interim.