WEBVTT NOTE This file was generated by Descript 00:00:00.000 --> 00:00:03.240 Judith: Welcome to Berry's In the Interim podcast, where we explore the 00:00:03.240 --> 00:00:07.230 cutting edge of innovative clinical trial design for the pharmaceutical and 00:00:07.230 --> 00:00:09.409 medical industries, and so much more. 00:00:09.720 --> 00:00:10.409 Let's dive in. 00:00:12.237 --> 00:00:15.367 Well, welcome everybody back to In The Interim. 00:00:15.607 --> 00:00:17.177 I'm your host, Scott Berry. 00:00:17.887 --> 00:00:20.317 Appreciate you joining me again today. 00:00:20.807 --> 00:00:26.797 Uh, In The Interim is a podcast of all things clinical trial science. 00:00:27.507 --> 00:00:34.337 Now, if you go back to some of this, I, I try, I, I try to create a, a, 00:00:34.337 --> 00:00:41.357 a much more welcome intro to each of the topics, and it might be a story of 00:00:41.367 --> 00:00:45.717 something that happens that I feel like is relevant to introduce the story. 00:00:46.417 --> 00:00:51.927 So I, I was, I was talking with, uh, uh, my family, I was talking to my wife, 00:00:51.927 --> 00:00:57.277 Tammy, who she ends up being a fairly common topic on this, this podcast. 00:00:57.777 --> 00:01:02.637 And I was describing the, the topic and thinking about is, you know, is there 00:01:02.637 --> 00:01:07.707 something, there's gotta be something where, um, r- really the topic is 00:01:08.017 --> 00:01:14.137 where something is true in a particular scenario and people understand that, 00:01:14.677 --> 00:01:18.687 but it's not true in another scenario, but they carry that with them and 00:01:18.687 --> 00:01:23.917 there's a sort of myth about it that this is a, a much more universal thing. 00:01:24.207 --> 00:01:30.147 And, um, and not understanding the aspect of it, and this just carries forward. 00:01:30.587 --> 00:01:33.837 And was, was talking about superstitions, for example. 00:01:34.127 --> 00:01:34.987 Uh, we were… 00:01:35.227 --> 00:01:39.237 I was, I was talking about we sit at our, our son's baseball game, and we 00:01:39.237 --> 00:01:42.907 might not say a word at the baseball game, but then when we're home at the TV, 00:01:43.167 --> 00:01:46.897 we might not say a word thinking that, you know, th- th- this is different. 00:01:47.357 --> 00:01:50.737 So I was introducing this, we were thinking about many of this, and I'll 00:01:50.737 --> 00:01:53.227 introduce exactly what I mean by this. 00:01:53.577 --> 00:01:56.757 And I said, "Well, you know what, what I'm trying to describe is there's 00:01:56.767 --> 00:02:04.999 this myth in clinical trials that looking at data Causes penalties. 00:02:06.099 --> 00:02:09.709 And, uh, I'll, I'll sort of explain more of this. 00:02:10.079 --> 00:02:12.989 And there, there's this, this thought that, uh, uh… 00:02:12.989 --> 00:02:15.559 And it is true in some scenarios. 00:02:15.859 --> 00:02:21.639 In some actions you're going to take in the trial, you have to adjust 00:02:21.679 --> 00:02:25.489 alpha because of those actions. 00:02:26.339 --> 00:02:28.849 But it depends on the action you're gonna take. 00:02:29.089 --> 00:02:33.939 And in other settings, you, you could do a million futility analyses and there's no 00:02:33.999 --> 00:02:36.619 adjustment to alpha through, through it. 00:02:36.669 --> 00:02:41.149 But there's this perception that looking at data causes penalties 00:02:41.149 --> 00:02:44.459 of, you know, what, what's a, a common thing in asking her ways 00:02:44.459 --> 00:02:46.199 that I could introduce this topic. 00:02:46.199 --> 00:02:52.369 And she, she was thinking about ways, and then she said to me, "Haven't you 00:02:52.369 --> 00:02:55.319 done that already on your podcast? 00:02:55.359 --> 00:02:58.459 Don't you do that, you know, multiple times? 00:02:58.459 --> 00:03:00.069 Are you gonna talk about that again?" 00:03:00.519 --> 00:03:02.869 So, uh, and it is true. 00:03:03.269 --> 00:03:04.319 And, but it… 00:03:04.369 --> 00:03:06.309 I was triggered again today. 00:03:06.309 --> 00:03:12.509 I was doing a, a call with a potential new client, and they brought up the 00:03:12.509 --> 00:03:17.319 idea in their trial, uh, enrollment is slow and, and various issues. 00:03:17.319 --> 00:03:18.209 I won't go into it. 00:03:18.209 --> 00:03:21.489 I don't, I don't want to give away any confidential information. 00:03:21.489 --> 00:03:25.779 But the topic came up that they were thinking about doing futility 00:03:25.779 --> 00:03:30.869 analysis in the trial, and they didn't wanna do it because they thought 00:03:30.909 --> 00:03:32.489 they would have to adjust alpha. 00:03:33.409 --> 00:03:39.509 And this overall perception largely that looking at data is a bad thing 00:03:39.529 --> 00:03:44.989 to do, that that sort of i- is a corollary to that, and it comes to this. 00:03:45.299 --> 00:03:48.209 This is almost an everyday occurrence. 00:03:48.739 --> 00:03:53.809 And so when my wife was trying to come up with me with, with ways to introduce 00:03:53.809 --> 00:03:56.179 this topic, in a way, it actually… 00:03:56.689 --> 00:04:00.779 The act of trying to come up with a story is my intro to the topic 00:04:00.779 --> 00:04:04.549 today, where, uh, uh, you know, "Haven't you done this before?" 00:04:04.909 --> 00:04:06.499 Uh, and I have done this before. 00:04:06.499 --> 00:04:07.859 It's a little bit different today. 00:04:07.869 --> 00:04:10.559 I'm gonna come at this in a different way today. 00:04:10.979 --> 00:04:14.819 But I wanna get at this topic and… 00:04:14.909 --> 00:04:20.449 of, of adaptive actions in trials and what that means. 00:04:20.479 --> 00:04:23.619 And there's actually beautiful mathematical aspects to this, 00:04:23.629 --> 00:04:24.729 so I wanna get into that. 00:04:25.119 --> 00:04:32.419 The adaptive action matrix in trials, and it's really, it, it's really kind of a 00:04:32.419 --> 00:04:37.549 beautiful thing, and I don't think it's widely understood, that aspect of it. 00:04:37.549 --> 00:04:43.085 So let's back up a little bit and think about how does this myth Uh, 00:04:43.185 --> 00:04:47.515 uh, come about and why do people still believe that looking at data 00:04:47.965 --> 00:04:52.565 somehow is a penalty i-i-in the trial? 00:04:53.245 --> 00:04:57.445 Historically, really the only adaptations that were done in 00:04:57.445 --> 00:05:00.865 trials were to look for superiority. 00:05:01.555 --> 00:05:03.575 Group sequential designs. 00:05:03.655 --> 00:05:08.775 So at seventy-five percent or fifty percent of the way through the trial, we 00:05:08.775 --> 00:05:13.235 would do an interim analysis, and if the data were promising enough, and we would 00:05:13.235 --> 00:05:20.615 have to adjust alpha for that because you can't do a, a two point five percent test 00:05:20.615 --> 00:05:26.245 multiple times and still have two point five percent error in a superiority test. 00:05:26.245 --> 00:05:29.165 And so we have to adjust alpha for the multiple looks. 00:05:29.415 --> 00:05:32.505 That was really the only adaptations that were done. 00:05:32.785 --> 00:05:37.615 Simple designs in this, and we had the really beautiful mathematics of 00:05:37.615 --> 00:05:43.095 alpha spending functions, whether it's, it's Pocock or Brian Fleming, Kim 00:05:43.095 --> 00:05:45.065 De Mets, whatever spending function. 00:05:45.375 --> 00:05:48.765 We had this beautiful math around it, th-this adjustment. 00:05:49.095 --> 00:05:51.645 And so these are the only times it was done. 00:05:51.645 --> 00:05:55.995 And so we'd be running large phase three trials, and this is it. 00:05:56.345 --> 00:06:00.685 And with that came the notion that, okay, every time we look at data, 00:06:00.835 --> 00:06:02.465 you have to make an adjustment. 00:06:03.055 --> 00:06:04.325 And it's not true. 00:06:04.845 --> 00:06:09.095 When you do an interim analysis for superiority, you 00:06:09.095 --> 00:06:10.615 have to make an adjustment. 00:06:10.645 --> 00:06:12.445 And those were the only times it was done. 00:06:12.445 --> 00:06:15.735 And so there was this general perception that, okay, every time you do that. 00:06:15.935 --> 00:06:23.155 And it got attached to looking at data, not the adaptive action of 00:06:23.395 --> 00:06:29.405 claiming superiority at the time, which does involve an alpha adjustment. 00:06:30.565 --> 00:06:36.715 And hence it's still ubiquitous, and I, I, I heard it today, and they were 00:06:36.715 --> 00:06:42.095 surprised when I told them, "Oh, you could do lots of i-interims for futility and no 00:06:42.095 --> 00:06:45.225 adjustment needed, uh, in that setting." 00:06:45.595 --> 00:06:46.655 "Oh, that's great. 00:06:46.755 --> 00:06:48.145 We, we should do that." 00:06:48.185 --> 00:06:52.095 Yeah, you should do that, um, i-in the setting. 00:06:52.095 --> 00:06:58.825 And so it's this, this, this notion that looking at data means you need to adjust. 00:06:58.855 --> 00:06:59.945 There's penalties to it. 00:06:59.945 --> 00:07:00.865 It's bad to do it. 00:07:01.055 --> 00:07:04.255 That's the other corollary to do that, that it's bad to do it. 00:07:04.795 --> 00:07:06.075 So Tammy is right. 00:07:06.085 --> 00:07:06.995 My wife is right. 00:07:06.995 --> 00:07:10.685 I did a podcast on this, and she even says something like spending alpha. 00:07:11.035 --> 00:07:17.705 So episode sixteen was spending alpha, and, uh, it was, last year it was the 00:07:17.705 --> 00:07:24.135 most listened to, watched podcast that I did somewhat for a Bayesian statistician. 00:07:24.385 --> 00:07:27.155 Spending alpha was one of the more common ones. 00:07:27.535 --> 00:07:30.385 Um, so you may be interested in going back to that. 00:07:30.765 --> 00:07:34.325 I want to talk about something a little bit different, uh, in that, 00:07:34.615 --> 00:07:39.945 and that is the different adaptive actions and potential implications 00:07:41.539 --> 00:07:46.869 There's a really, really cool aspect to this, and I'll, I'll go into it. 00:07:46.909 --> 00:07:49.999 And, and so I want you to envision a matrix. 00:07:50.979 --> 00:07:55.429 I want you to envision the matrix where the columns… 00:07:56.389 --> 00:08:00.259 Uh, by the way, one fun thing of doing a podcast, and it's a really 00:08:00.329 --> 00:08:08.219 good, um, activity to have to do, is to do this podcast knowing people are 00:08:08.219 --> 00:08:10.999 listening, and I can't use slides. 00:08:11.989 --> 00:08:16.639 It's, uh, we, we use slides become a crutch, a graphic around 00:08:16.649 --> 00:08:18.299 that, and they're wonderful. 00:08:18.489 --> 00:08:20.209 Uh, if ability to it. 00:08:20.219 --> 00:08:24.589 But the ability to have to explain something without slides, I think is 00:08:24.589 --> 00:08:29.879 a, is a good activity to go through in thinking about how to communicate ideas. 00:08:30.689 --> 00:08:33.099 Uh, it, it makes me a beg- better communicator. 00:08:33.099 --> 00:08:35.819 I don't know about a good communicator, but I, I think it's helpful. 00:08:36.129 --> 00:08:41.529 So I want you to think about the columns here as the data. 00:08:42.119 --> 00:08:49.939 So we're gonna do an interim analysis, and the columns are positive data. 00:08:50.729 --> 00:08:54.069 And I'll come back to what does it mean to be positive data. 00:08:55.209 --> 00:09:00.129 So when the data are positive is the right-hand column. 00:09:01.529 --> 00:09:07.495 The left-hand column is when the data are negative Okay. 00:09:07.865 --> 00:09:11.095 And so I'm gonna differentiate when you do an interim when 00:09:11.095 --> 00:09:12.935 the data are in these cases. 00:09:14.135 --> 00:09:25.565 And then the, the rows here are the action, and almost every action you take 00:09:25.565 --> 00:09:33.985 at a, at an interim has an implication to the effective sample size in your trial. 00:09:35.375 --> 00:09:40.275 And think about it that you increase the sample size or you decrease 00:09:40.315 --> 00:09:42.515 the sample size at the interim. 00:09:43.085 --> 00:09:47.495 And I'll talk at that it's not explicitly that you change N. 00:09:47.765 --> 00:09:54.115 Sometimes you take other actions that have an implication to your sample size. 00:09:55.305 --> 00:09:57.945 So here's our two-by-two matrix. 00:09:58.965 --> 00:10:04.225 The columns are positive data on the right, negative data on the left. 00:10:05.755 --> 00:10:11.235 The rows on top are increased sample size, and on the bottom, 00:10:11.235 --> 00:10:12.795 you decrease sample size. 00:10:12.805 --> 00:10:14.795 So it's this two-by-two matrix. 00:10:16.595 --> 00:10:23.235 The really cool thing about adaptive actions and do we need to adjust 00:10:23.235 --> 00:10:30.475 alpha, does it cause the need to adjust alpha, is depends on which 00:10:30.475 --> 00:10:33.285 of those four quadrants you're in. 00:10:34.925 --> 00:10:41.275 So we talked about the historical scenario where when your data are 00:10:41.295 --> 00:10:44.575 good, you decrease the sample size. 00:10:44.585 --> 00:10:49.855 So if we're doing a standard group sequential trial where a hundred is 00:10:49.855 --> 00:10:54.305 your maximum sample size, this is a generic number, and you do an interim 00:10:54.305 --> 00:10:58.295 at fifty patients or seventy-five patients, let's do fifty patients, 00:10:58.825 --> 00:11:05.765 and we look to see if the data are, uh, below a adjusted P value, .001. 00:11:05.765 --> 00:11:13.095 If it is, so fifty patients is when our data are really good, we 00:11:13.095 --> 00:11:15.735 decrease from a hundred to fifty. 00:11:17.125 --> 00:11:20.595 That's the quadrant where our data are positive. 00:11:20.835 --> 00:11:23.055 We decrease the sample size. 00:11:23.275 --> 00:11:28.025 You have to adjust for actions in that quadrant. 00:11:28.605 --> 00:11:33.625 You are inflating type one error when you are in that quadrant, and 00:11:33.625 --> 00:11:37.905 that's the one that every adaptive design lived in that quadrant. 00:11:38.435 --> 00:11:43.015 And everybody thought, "Okay, everything in all three quadrants, you have to 00:11:43.015 --> 00:11:48.075 adjust alpha," but just by looking at data, and that's not the case. 00:11:48.675 --> 00:11:56.635 So, uh, in that, in that, uh, quadrant, the most common one of these, of 00:11:56.675 --> 00:11:59.535 course, is group sequential designs. 00:11:59.785 --> 00:12:04.767 And yes, you have to adjust alpha for that Okay. 00:12:05.547 --> 00:12:08.187 And we, we, we have other quadrants. 00:12:08.197 --> 00:12:16.087 So let's think about the other most common quadrant is when your data are negative 00:12:18.549 --> 00:12:21.339 and you decrease sample size 00:12:23.555 --> 00:12:27.645 So, and we all understand that that's futility. 00:12:28.865 --> 00:12:33.395 So when your data are negative and you're gonna decrease sample size, 00:12:34.415 --> 00:12:36.635 you actually deflate type I error. 00:12:37.875 --> 00:12:42.025 You take away the shot that you might have won had you gone to 100. 00:12:42.285 --> 00:12:46.515 So at 50 patients, when our data aren't very good, we stop for futility 00:12:46.545 --> 00:12:49.085 unlikely, and the trial loses. 00:12:49.455 --> 00:12:56.145 You only lessen the chance of success, and under the null hypothesis, 00:12:56.145 --> 00:13:01.825 you lessen the chance of getting an incorrect positive result. 00:13:02.755 --> 00:13:04.225 That's type I error. 00:13:04.685 --> 00:13:06.595 Um, in that case, we lessen it. 00:13:06.615 --> 00:13:08.705 We actually deflate type I error. 00:13:09.145 --> 00:13:14.545 So in this quadrant, the lower right, when the data are positive and we decrease the 00:13:14.545 --> 00:13:17.295 sample size, we increase type I error. 00:13:17.685 --> 00:13:23.625 But when you move over a quadrant to the left of that, when the, when the 00:13:23.635 --> 00:13:28.845 data are negative and y- your action is to decrease the sample size, you 00:13:28.875 --> 00:13:32.517 deflate type I error In that setting. 00:13:33.507 --> 00:13:38.537 Now, there's-- it's controversial as to whether by doing futility, 00:13:38.557 --> 00:13:43.947 we get to actually buy back alpha, and that a, a, a different topic 00:13:43.947 --> 00:13:46.307 for a different, different day. 00:13:46.897 --> 00:13:51.987 Um, uh, in that setting, generally, that, that's generally not done, but 00:13:51.987 --> 00:13:54.017 in some s- cases, it might be done. 00:13:54.657 --> 00:13:56.947 But that's, that's the result of it. 00:13:56.957 --> 00:14:03.887 So there's a circumstance where the advice I gave today was you can do interims 00:14:03.897 --> 00:14:08.047 for futility over and over and over and over again, and you can't inflate type 00:14:08.127 --> 00:14:15.247 I error because the only action you can take in that scenario is to decrease the 00:14:15.247 --> 00:14:17.227 sample size when the data are negative. 00:14:17.287 --> 00:14:21.627 Now, the important thing is, and I, I, I described that we would talk 00:14:21.637 --> 00:14:25.097 about, what does it mean for the data to be positive or negative? 00:14:26.247 --> 00:14:32.287 Now, it can depend by the action, but largely when the data, when the data 00:14:32.317 --> 00:14:37.677 are being negative, meaning at that point, we do not have superiority. 00:14:38.507 --> 00:14:41.617 And so that's-- doesn't have to be all that negative. 00:14:41.647 --> 00:14:44.937 The data can still be on the positive side. 00:14:45.467 --> 00:14:51.527 So for these two a- two adaptive actions, stopping early for supe- superiority 00:14:52.017 --> 00:14:56.197 or stopping for futility, the threshold is actually do we have significance 00:14:56.337 --> 00:14:58.737 at that time point, uh, within that? 00:14:58.787 --> 00:15:02.677 I mean, that's kind of the, the, the break point of this as to what 00:15:02.687 --> 00:15:04.697 data are negative and positive. 00:15:06.017 --> 00:15:11.077 Fairly straightforward, um, actions in these settings. 00:15:11.607 --> 00:15:16.037 Now, it gets really interesting when you go to the other two quadrants. 00:15:17.117 --> 00:15:22.217 So let's think about the quadrant where the data are positive, 00:15:23.387 --> 00:15:25.487 and you increase the sample size 00:15:28.323 --> 00:15:31.983 So, so what, what, what kinds of actions do that? 00:15:34.067 --> 00:15:38.967 In a phase III trial, an action that does that is the promising zone. 00:15:40.547 --> 00:15:43.487 So I want to spend a little bit of time on that 'cause I think that's 00:15:43.507 --> 00:15:45.907 the beauty of the mathematics of it. 00:15:46.627 --> 00:15:52.747 But other actions at that time are one common adaptive action is 00:15:52.787 --> 00:15:55.007 response adaptive randomization. 00:15:56.587 --> 00:16:04.227 So suppose you have two doses and a control, and we're gonna-- when the 00:16:04.447 --> 00:16:10.547 data's positive on one of the two doses, we increase the randomization to it. 00:16:11.047 --> 00:16:14.007 We're increasing its effective sample size. 00:16:14.887 --> 00:16:18.947 When its data are positive, we increase its effective sample size. 00:16:20.387 --> 00:16:21.847 You decrease type I error. 00:16:22.867 --> 00:16:27.267 In this matrix, the bottom right was, uh, increasing type I error. 00:16:27.487 --> 00:16:32.087 As you move from bottom right to upper right, you decrease type I error. 00:16:32.447 --> 00:16:35.247 You're gonna see that in the bottom right, we increase type I error, 00:16:35.307 --> 00:16:38.247 in the upper left, we increase type I error, and then the two 00:16:38.366 --> 00:16:40.667 off-diagonals, we decrease type I error. 00:16:40.787 --> 00:16:42.367 Futility decreases type I error. 00:16:43.057 --> 00:16:49.527 Response adaptive randomization when the data are positive on the dose, 00:16:49.547 --> 00:16:56.547 and we increase its allocation for the r- for the next stage or moving 00:16:56.587 --> 00:16:58.337 forward, we decrease type I error. 00:16:58.747 --> 00:17:01.787 Response adaptive randomization decreases type I error. 00:17:01.867 --> 00:17:06.107 Now, there are details to that about what, what are we, what are we testing 00:17:06.187 --> 00:17:09.787 in that scenario, but if we're testing are one of the doses better than 00:17:09.827 --> 00:17:15.547 placebo and we're increasing the rate to that, we decrease type I error. 00:17:16.127 --> 00:17:21.567 You don't have to i-in-- change your alpha level to do response adaptive 00:17:21.627 --> 00:17:23.707 randomization a hundred times in a trial. 00:17:25.207 --> 00:17:26.247 No adjustment needed. 00:17:26.987 --> 00:17:34.307 We've had special protocol assessments with the FDA where they agree, and we, 00:17:34.407 --> 00:17:38.187 we did twenty interim analyses to do response adaptive randomization, and 00:17:38.207 --> 00:17:39.967 there's no inflation of type I error. 00:17:39.967 --> 00:17:45.107 error And so that's a really neat thing that you don't necessarily 00:17:45.147 --> 00:17:48.827 think about in adaptive designs and what are the actions in that. 00:17:49.427 --> 00:17:52.527 Other things of even dropping arms in a trial. 00:17:52.587 --> 00:17:56.767 Suppose you have a h- a hundred patients in a trial, and when you 00:17:56.847 --> 00:18:00.347 get to fifty, you have two doses, we're doing one-to-one-to-one 00:18:00.387 --> 00:18:03.387 randomization, and we drop a dose. 00:18:04.327 --> 00:18:08.087 And the remaining fifty patients, we don't change the overall sample size, 00:18:08.407 --> 00:18:12.447 the remaining fifty patients that would have been one-to-one-to-one are now 00:18:12.507 --> 00:18:14.447 one-to-one 'cause we dropped a dose. 00:18:15.047 --> 00:18:18.667 We increase the sample size on the remaining dose. 00:18:19.127 --> 00:18:24.187 By dropping that arm, we deflate type I error in that scenario. 00:18:24.247 --> 00:18:29.557 So r-- and by the way, arm dropping is a form of response adaptive randomization 00:18:30.563 --> 00:18:37.073 By making its probability zero in that we're changing its randomization based 00:18:37.073 --> 00:18:40.223 on the data, uh, in that scenario. 00:18:41.493 --> 00:18:46.323 So tho- that, that's kind of a weird place within this matrix that we don't 00:18:46.403 --> 00:18:48.173 spend a lot of time talking about. 00:18:49.433 --> 00:18:52.603 I wanna go back to the promising zone 'cause it brings out really 00:18:52.683 --> 00:18:59.093 interesting mathematics of this, where the, the-- what is the promising zone? 00:18:59.653 --> 00:19:09.493 The promising zone is an, is a adaptive sample size trial that, um, uh, 00:19:09.493 --> 00:19:11.173 uh, I'll give you the basics of it. 00:19:11.253 --> 00:19:12.643 We're going for, we're… 00:19:12.653 --> 00:19:15.683 for a sample size of a hundred one-to-one randomized trial. 00:19:15.943 --> 00:19:18.783 The planned sample size is a hundred. 00:19:20.103 --> 00:19:25.343 We're gonna do an interim before a hundred at fifty, say. 00:19:26.563 --> 00:19:33.003 And based on the data at fifty, we might continue on to the planned one hundred. 00:19:33.783 --> 00:19:38.683 But at that time, we might increase the sample size to be bigger than a hundred. 00:19:40.663 --> 00:19:45.483 Eh, this is a kind of sample size re-estimation technique, this special one. 00:19:46.443 --> 00:19:51.253 There's a really beautiful result mathematically, and this comes from 00:19:51.253 --> 00:20:00.433 Mehta and Pocock paper, where when you do the look at fifty, if the conditional 00:20:00.523 --> 00:20:05.043 power for a hundred is above 50 00:20:07.875 --> 00:20:12.405 You can increase the sample size above a hundred to anything you want, 00:20:12.685 --> 00:20:14.865 and you don't increase type I error. 00:20:14.875 --> 00:20:17.125 You actually decrease type I error 00:20:19.299 --> 00:20:24.229 So that's the beautiful result where I describe-- I, I said when the data are 00:20:24.289 --> 00:20:28.129 positive, you increase the sample size. 00:20:28.559 --> 00:20:34.159 This mathematical result, so when you're sitting there at 50, if your data are 00:20:34.199 --> 00:20:40.339 trending to winning at 100, conditional power of 50 means you're right on the ef- 00:20:40.359 --> 00:20:42.889 the effect size you need to win at 100. 00:20:43.239 --> 00:20:46.929 If you continue on with that effect size, you're gonna win. 00:20:47.629 --> 00:20:49.449 If you get a little bit worse, you're gonna lose. 00:20:49.449 --> 00:20:50.769 Anything better, you're gonna win. 00:20:50.969 --> 00:20:52.769 That's conditional power of 50. 00:20:52.769 --> 00:20:55.719 Right now, we're at the threshold we need to win. 00:20:56.479 --> 00:21:02.239 That's mathematically the definition of positive data when you're sitting 00:21:02.239 --> 00:21:04.829 at 50 and you're planning 100. 00:21:05.069 --> 00:21:07.029 You could increase it to anything you want. 00:21:07.149 --> 00:21:11.459 So this design allows you, based on as long as your conditional 00:21:11.469 --> 00:21:17.769 power is above 50, you can increase your sample size to get 80% power. 00:21:17.779 --> 00:21:22.529 You can increase it to 200 to get 80% power within the scenario. 00:21:23.689 --> 00:21:25.899 Notice a couple things of this. 00:21:26.929 --> 00:21:32.239 In this matrix of positive data and increasing your sample size, if you're 00:21:32.239 --> 00:21:38.419 right on the line of positive data, so you're at 50% conditional power, 00:21:38.629 --> 00:21:40.839 you don't lose any alpha in that. 00:21:41.159 --> 00:21:44.429 But if your conditional power is above that, you're actually now spending alpha. 00:21:44.609 --> 00:21:50.929 I, I'm sorry, you're losing alpha, like futility, that the promising 00:21:50.929 --> 00:21:56.389 zone design loses alpha, uh, i- in that because of this action. 00:21:56.869 --> 00:22:02.779 Because at 50, you're increasing to a sample size, but you can only do it in 00:22:02.779 --> 00:22:06.489 a very narrow region, uh, of the design. 00:22:06.759 --> 00:22:11.059 So a separate podcast, my Tammy may be listening at this and say, "Ooh, 00:22:11.099 --> 00:22:12.629 I've heard this somewhere before." 00:22:12.869 --> 00:22:15.309 By the way, she does listen to each of the podcasts. 00:22:15.709 --> 00:22:21.199 Um, I, I did a podcast, episode 31, the not so promising zone 00:22:21.199 --> 00:22:27.309 design It's, it's commonly done, and it doesn't work very well. 00:22:27.419 --> 00:22:29.159 So please go into that one. 00:22:29.159 --> 00:22:33.809 So the math is beautiful, fantastic. 00:22:34.289 --> 00:22:37.439 The design and the actions are not. 00:22:37.829 --> 00:22:39.489 It doesn't work very well. 00:22:39.899 --> 00:22:45.499 If you compare it to other adaptive approaches, it doesn't work very well. 00:22:45.689 --> 00:22:47.859 Spend alpha and do a group sequential. 00:22:48.019 --> 00:22:50.769 That's the spending alpha episode 16. 00:22:50.779 --> 00:22:53.039 So it all kind of ties in together. 00:22:53.359 --> 00:22:53.649 I got… 00:22:53.899 --> 00:22:56.069 You know, so I don't wanna go deep into those. 00:22:56.099 --> 00:23:00.089 Please join those episodes if, if, if any of that is interesting. 00:23:00.949 --> 00:23:06.279 So the promising zone design lives in this new quadrant that we haven't been 00:23:06.279 --> 00:23:08.309 doing many adaptive designs in there. 00:23:08.929 --> 00:23:14.819 Now, the, the interesting thing is they've created ways that you can solve exactly. 00:23:14.829 --> 00:23:17.859 Sometimes you can push that fifty percent even lower. 00:23:18.439 --> 00:23:22.139 You know, forty percent conditional power thirty-seven without inflating 00:23:22.139 --> 00:23:24.689 type one error, uh, in that scenario. 00:23:24.689 --> 00:23:29.439 So you can calculate the math, and then if you go below that, where your 00:23:29.449 --> 00:23:33.629 data is defined to be negative, we actually move the quadrant from the 00:23:33.629 --> 00:23:39.549 upper right to the upper left, where our data at that time are negative, 00:23:40.089 --> 00:23:42.379 and we increase the sample size. 00:23:43.639 --> 00:23:45.999 That increases type one error. 00:23:47.439 --> 00:23:52.199 So now we've got this matrix of actions and data. 00:23:53.099 --> 00:23:54.649 And re-remind us where we are. 00:23:54.659 --> 00:23:57.339 The, the, the right-hand column is positive data. 00:23:58.539 --> 00:24:02.709 If you decrease the sample size or you do an action that decreases sample 00:24:02.709 --> 00:24:08.005 size, you increase type one error If you do something that increases 00:24:08.005 --> 00:24:12.535 the sample size when you have good data, that decreases the type one 00:24:12.535 --> 00:24:14.245 error in those several actions. 00:24:15.055 --> 00:24:18.805 Moving over to the left-hand column are negative data. 00:24:19.905 --> 00:24:26.495 And so now to win at 100, we're actually in this conditional power less than 50%. 00:24:27.395 --> 00:24:31.645 We're, we're, we- we're, we're not trending to a successful trial. 00:24:31.645 --> 00:24:32.865 Our data are negative. 00:24:33.385 --> 00:24:37.775 What if we wanna push the sample size to 200 or, or to 1,000? 00:24:38.565 --> 00:24:42.595 Now, when the data are negative, if you increase your sample size, you're almost 00:24:42.595 --> 00:24:44.225 saying, "Oh, just give me a new trial. 00:24:44.655 --> 00:24:45.885 Give me 1,000 patients. 00:24:45.885 --> 00:24:46.815 Give me bigger numbers." 00:24:46.975 --> 00:24:51.195 You're almost taking multiple shots on goal by doing that. 00:24:51.415 --> 00:24:54.885 When it's negative, I'll just collect enough data that will, will, will, 00:24:54.895 --> 00:24:57.075 will weigh out this initial negative. 00:24:57.405 --> 00:25:00.225 You're, you're doing kind of the opposite of group sequential, but 00:25:00.225 --> 00:25:04.255 you have the same effect of it, that you're getting multiple shots on goal. 00:25:04.565 --> 00:25:07.335 You increase type one error by doing that. 00:25:07.995 --> 00:25:13.545 This is not understand- stood very well that the, the-- this circumstance. 00:25:13.545 --> 00:25:18.405 There are a lot of people that I see designs where, oh, we'll just do an 00:25:18.415 --> 00:25:24.685 interim at the design, and we'll push the sample size to be whatever sample size 00:25:24.695 --> 00:25:26.675 gives us ninety percent conditional power. 00:25:27.895 --> 00:25:32.855 That trial will inflate type one error i- in the scenario. 00:25:32.865 --> 00:25:34.615 You, you, you don't get this free lunch. 00:25:34.615 --> 00:25:38.585 So if you're, if you're stuck on type one error, you live in that 00:25:38.595 --> 00:25:41.065 quadrant there of doing that. 00:25:41.065 --> 00:25:43.575 A group sequential design, you could almost think of group 00:25:43.575 --> 00:25:48.005 sequential as the opposite of we have 100 and we might stop at 50. 00:25:48.345 --> 00:25:52.175 A group sequential design is we're gonna go to 50 and see if we win. 00:25:52.825 --> 00:25:58.415 If we don't win, we're gonna increase the sample size to 75 and see if we win. 00:25:58.415 --> 00:26:02.825 And oh, if our data are negative, we're gonna go to 100 and look at it. 00:26:03.155 --> 00:26:08.005 You can rewrite a group sequential design to live in this quadrant in the upper 00:26:08.005 --> 00:26:14.705 left, where we're taking actions on negative data to increase our sample size. 00:26:15.605 --> 00:26:20.895 Multiple shots on goal, you're doing things that increase type one error 00:26:24.043 --> 00:26:24.603 Okay. 00:26:25.693 --> 00:26:28.563 So now this is the, the, this is the circumstance. 00:26:28.573 --> 00:26:33.723 Now overall, the general understanding of this is because we're in 00:26:33.723 --> 00:26:36.993 the lower right-hand quadrant, that looking at data is bad. 00:26:37.593 --> 00:26:43.813 There are huge tools throughout here to create a better design, and going back to 00:26:43.813 --> 00:26:50.853 this episode, uh, 16 on spending alpha, spending alpha is a good thing to do. 00:26:51.723 --> 00:26:58.353 Recognizing your, your actions here and when it happens in the data 00:26:58.363 --> 00:27:03.683 in order to create a better design to get better answers within the 00:27:03.713 --> 00:27:06.653 constraints of it is a good thing to do. 00:27:08.183 --> 00:27:12.903 So I hope envisioning all of this as your actions, the important 00:27:12.913 --> 00:27:17.493 part of it is looking at data doesn't increase type one error. 00:27:17.503 --> 00:27:19.493 It's the action you may take. 00:27:20.863 --> 00:27:26.003 Really important part of adaptive designs, this is why it's absolutely critical 00:27:26.173 --> 00:27:28.413 you have a prospective adaptive design. 00:27:29.303 --> 00:27:35.603 You get a great deal of flexibility in the design as long as it's pre-specified. 00:27:36.673 --> 00:27:40.723 If you try to go with an adaptive design as, "Oh, I'll just look at 00:27:40.723 --> 00:27:46.363 the data and then decide what to do," you can't argue you're in this 00:27:46.373 --> 00:27:49.863 quadrant, you're in that quadrant, these are the actions I'm taking. 00:27:50.313 --> 00:27:54.583 You can't even control type one error from such a design because you haven't 00:27:54.593 --> 00:28:02.949 defined the actions at that time from it The other, the other thing that I, I find 00:28:03.119 --> 00:28:08.699 particularly annoying, uh, my pet peeve of the day, Kurt Veily has pet peeves of the 00:28:08.699 --> 00:28:15.939 day, is I'll see where somebody's doing a futility analysis, and they spend .0001 00:28:15.939 --> 00:28:20.739 alpha as a-- But are you gonna do a superiority test? 00:28:20.749 --> 00:28:23.279 No, no, we don't wanna do superiority at all. 00:28:23.579 --> 00:28:29.869 But they're throwing alpha to the gods because they almost think they have to. 00:28:30.589 --> 00:28:35.669 And it's almost disingenuous to do that because you don't wanna talk for-- stop 00:28:35.669 --> 00:28:39.459 for superiority, but yet you allocate it as though you might do something else. 00:28:39.769 --> 00:28:41.409 What, what, what's the design? 00:28:41.419 --> 00:28:45.309 I think it's a circumstance where you just be very clear, be prospective. 00:28:45.769 --> 00:28:47.979 You can have a great deal of flexibility. 00:28:47.989 --> 00:28:53.149 You can design better trials by being fully prospective in it. 00:28:53.169 --> 00:28:55.749 A great deal of flexibility in the sample size. 00:28:56.299 --> 00:28:59.749 Uh, you can accomplish what you're trying to accomplish with, with 00:28:59.849 --> 00:29:04.209 efficient trial designs, but it has to be prospectively set up, and you can 00:29:04.209 --> 00:29:06.639 take advantage of some of the math here. 00:29:07.269 --> 00:29:11.309 Um, going back to introducing this that, uh, Tammy may say, 00:29:11.309 --> 00:29:12.649 "I've heard all that before." 00:29:13.099 --> 00:29:18.069 Uh, hoping this matrix idea of thinking about it, data positive, data negative, 00:29:18.069 --> 00:29:23.459 and the action being increasing effective sample size or decreasing effective sample 00:29:23.459 --> 00:29:29.279 size might be something new for you to think about the world of adaptive designs. 00:29:29.539 --> 00:29:33.909 I, I was going to go into this whole aspect of in golf, knowing the 00:29:33.909 --> 00:29:37.979 rules in golf, you can actually drop your ball in certain situations. 00:29:37.979 --> 00:29:42.239 You get a great deal of flexibility under the rules of golf. 00:29:42.619 --> 00:29:45.869 Um, and it's not always you can't touch your ball, you can't do it. 00:29:45.889 --> 00:29:49.869 Knowing the rules helps you to be a better golfer. 00:29:49.909 --> 00:29:54.639 I was gonna enter with that, but I thought I would enter with, with my 00:29:54.639 --> 00:29:56.379 wife saying, "You've done this before." 00:29:56.729 --> 00:30:00.739 And yes, I've done it before, and it's in part because every 00:30:00.739 --> 00:30:04.599 day in my day job, I hear these kinds of things from statisticians 00:30:04.619 --> 00:30:07.009 and clinical trialists alike. 00:30:08.249 --> 00:30:14.789 So know the rules, design better trials, use them to your advantage. 00:30:16.109 --> 00:30:19.079 Efficient, good trial designs out there. 00:30:20.189 --> 00:30:27.669 And until the meantime, we will be here, maybe spending alpha, maybe not spending 00:30:27.669 --> 00:30:30.529 alpha, but we'll be here in the interim.