WEBVTT NOTE This file was generated by Descript 00:00:00.029 --> 00:00:03.539 Judith: Welcome to Berry's In the Interim podcast, where we explore the 00:00:03.539 --> 00:00:07.780 cutting edge of innovative clinical trial design for the pharmaceutical and 00:00:07.780 --> 00:00:10.149 medical industries, and so much more. 00:00:10.810 --> 00:00:11.500 Let's dive in. 00:00:12.621 --> 00:00:16.071 Well, welcome everybody back to "In The Interim." 00:00:16.421 --> 00:00:17.961 I'm your host, Scott Barry. 00:00:18.741 --> 00:00:24.941 And for today's topic, I feel a little bit like, uh, years ago, 00:00:26.101 --> 00:00:29.921 I was on faculty at Texas A&M. 00:00:30.551 --> 00:00:36.901 I was assistant professor of statistics at Texas A&M, and, uh, just starting 00:00:36.941 --> 00:00:40.791 at A&M, I had come from graduate school, came from Carnegie Mellon. 00:00:41.351 --> 00:00:46.241 And, uh, and I don't know if they still do it, but Texas A&M had a, at 00:00:46.241 --> 00:00:54.771 the time, a policy that every, um, graduate student, everybody pursuing a 00:00:54.811 --> 00:01:02.141 dissertation, would get a random faculty member on their dissertation committee 00:01:03.397 --> 00:01:08.657 So if you were a statistics graduate student doing a dissertation, you might 00:01:08.677 --> 00:01:16.257 get a faculty member from, uh, meat sciences or, uh, from something to be 00:01:16.257 --> 00:01:20.537 on your dissertation committee, a voting member of the dissertation committee. 00:01:21.137 --> 00:01:32.447 And so when I got there, I was assigned to a student's committee, and it was when I 00:01:32.447 --> 00:01:36.767 walked in, it was sort of like, "Oh, no." 00:01:37.447 --> 00:01:43.877 The, the-- generally, throughout all of the graduate programs, the 00:01:43.877 --> 00:01:49.727 worst thing that could happen to you was getting a statistician randomly 00:01:49.727 --> 00:01:52.997 assigned to your dissertation committee. 00:01:53.517 --> 00:01:59.157 So I think the first one I did was forestry and crop sciences, and 00:01:59.157 --> 00:02:00.697 they're actually doing experiments. 00:02:00.707 --> 00:02:01.837 It was, it was fascinating. 00:02:02.287 --> 00:02:07.167 And the second one I think was, was wildlife and fishery science. 00:02:07.197 --> 00:02:12.997 It was a fascinating project where they were following wolves, and they were 00:02:12.997 --> 00:02:17.017 tracking them, and they were reintroducing wolves into the wild, and they were 00:02:17.017 --> 00:02:18.897 trying to figure out their behaviors. 00:02:19.227 --> 00:02:20.647 It was dynamic data. 00:02:20.657 --> 00:02:25.147 It was absolutely fascinating, but they, they were fearful of a statistician 00:02:25.147 --> 00:02:28.997 walking in like, "Oh no, I now have a statistician on my committee." 00:02:30.117 --> 00:02:33.567 Now what, what does that have to do with today's topic? 00:02:33.917 --> 00:02:39.277 Today's topic, uh, I don't think it's the same sort of scenario because 00:02:39.287 --> 00:02:44.627 statisticians are rampant in the clinical trial, uh, uh, clinical trial world. 00:02:45.067 --> 00:02:53.657 But I am gonna talk about today a look at a recent in, i-issue of JAMA. 00:02:54.467 --> 00:03:00.237 So a wonderful thing about clinical trial design is you can go see the results, 00:03:00.247 --> 00:03:05.057 the published results of other trials and see those trials' design, see how they 00:03:05.057 --> 00:03:10.267 carry out the analyses, the interpretation of those, and of course, JAMA, New 00:03:10.267 --> 00:03:14.527 England Journal, Lancet, all of these provide this really nice opportunity. 00:03:15.107 --> 00:03:20.497 So I make sure whenever possible to open up these trials, to check out the 00:03:20.497 --> 00:03:24.057 design, see what they say, what's being done in the world of trial design, 00:03:24.087 --> 00:03:29.527 what, what would I have done differently in them, and do I agree with the 00:03:29.527 --> 00:03:32.447 interpretation from the medical journal? 00:03:32.457 --> 00:03:35.997 So I get, I get to jump in a statistician review, and I'm gonna 00:03:35.997 --> 00:03:37.757 talk about them on this show. 00:03:38.117 --> 00:03:40.937 So I always-- I, I've done, I've done one other. 00:03:40.937 --> 00:03:46.497 You can go back to the episode where I, I sort of randomly fell upon a really, 00:03:46.497 --> 00:03:54.621 really interesting trial of using, um iron supplements in heart failure where 00:03:54.621 --> 00:03:57.531 the, the patients are deficient in iron. 00:03:57.881 --> 00:03:59.841 Fascinating, fascinating trial. 00:03:59.841 --> 00:04:01.551 So you should go back to that episode. 00:04:01.971 --> 00:04:08.231 These trials, uh, I'm gonna look at two trials from read-- The email comes in 00:04:08.231 --> 00:04:13.511 from JAMA and I, I click on them and, you know, these are, these are interesting 00:04:13.511 --> 00:04:15.951 enough I, I, I wanna talk about these. 00:04:17.051 --> 00:04:21.701 Okay, so a little bit is doing this, doing this a bit publicly, and I hope people 00:04:21.701 --> 00:04:27.591 don't say, "Oh no, a, a statistician is reviewing my trials," uh, in this setting. 00:04:27.831 --> 00:04:31.061 By the way, an interesting thing, one of the m- more interesting things I 00:04:31.061 --> 00:04:37.521 got to do when I was at Texas A&M, I was a columnist for Chance Magazine. 00:04:37.651 --> 00:04:39.991 It's a publication of the ASA. 00:04:40.551 --> 00:04:47.291 And each quarter I got to write an article that was a statistician 00:04:47.291 --> 00:04:48.831 reads the sports pages. 00:04:49.461 --> 00:04:52.831 So I got to go in and find topics that were coming out, 00:04:52.831 --> 00:04:54.281 and I did this for 10 years. 00:04:54.281 --> 00:05:00.581 It was a quarterly magazine, so I wrote 40, uh, articles about reading the sports 00:05:00.591 --> 00:05:05.951 pages, things that come up in sports, uh, statistician's interpretation of them. 00:05:05.951 --> 00:05:07.721 So this is a little bit like that. 00:05:08.011 --> 00:05:14.091 This is a statistician reads JAMA and, and you know, what, what, what is, what, what 00:05:14.101 --> 00:05:16.591 do I look at when I read these trials? 00:05:17.891 --> 00:05:22.371 Okay, probably a bit of a bias in the trials that, that I look at, but 00:05:22.611 --> 00:05:26.761 this is a re- recent, uh, episode of what showed up in my mail. 00:05:27.711 --> 00:05:34.471 So the first trial was published June 4th, 2026, and it's called the 00:05:34.471 --> 00:05:37.381 TRACK Randomized Clinical Trial. 00:05:37.971 --> 00:05:45.431 It is a trial of, and the title of it is "Low-Dose Rivaroxaban in Cardiovascular 00:05:45.431 --> 00:05:48.291 Events in Advanced Kidney Disease." 00:05:49.001 --> 00:05:52.861 So a little bit about the trial, and I, I am not a clinician, so I, I'm, 00:05:52.911 --> 00:05:54.661 I'm reading the article of this. 00:05:55.031 --> 00:06:03.781 So rivaroxaban is a anticoagulant and apparently anticoagulations in some 00:06:03.781 --> 00:06:09.201 form of kidney disease with aspirin have demonstrated good clinical outcomes. 00:06:10.141 --> 00:06:19.681 So this is a trial looking at low-dose rivaroxaban in that looking at does it 00:06:19.731 --> 00:06:30.253 alter cardiovascular outcomes for patients with advanced chronic kidney disease Now 00:06:30.253 --> 00:06:37.703 these are, uh, CKD, chronic kidney disease stage four or five, including patients 00:06:37.703 --> 00:06:46.803 on dialysis, and they're randomized to, uh, a rivaroxaban low dose, uh, 00:06:46.833 --> 00:06:53.103 or not in the trial, and the primary outcome is a composite of cardiovascular 00:06:53.103 --> 00:06:55.533 death, non-fatal MI, stroke, uh, 00:06:58.193 --> 00:07:01.563 or peripheral artery disease event. 00:07:02.743 --> 00:07:06.793 And the primary safety outcome is major bleeding, which is typical 00:07:06.793 --> 00:07:08.773 of trials of anticoagulation. 00:07:09.073 --> 00:07:12.623 The risk, of course, are bleeding events, uh, in that. 00:07:13.793 --> 00:07:19.993 So that's the trial, the, the major part of what the trial is trying to address. 00:07:21.173 --> 00:07:24.223 Now, a little bit about the trial design. 00:07:24.223 --> 00:07:26.243 It's an event-driven trial. 00:07:27.413 --> 00:07:28.983 It, it's a little bit… 00:07:29.603 --> 00:07:32.343 I, I found it a little bit confusing in the trial design. 00:07:32.723 --> 00:07:38.753 It's an event-driven trial, but it talks about, uh, uh, the trial is 00:07:38.753 --> 00:07:44.613 designed to enroll nineteen hundred participants over three years with 00:07:44.613 --> 00:07:47.363 an expected follow-up of five years. 00:07:48.393 --> 00:07:55.413 So the nineteen hundred patients would provide statistical power to detect a 00:07:55.413 --> 00:08:00.833 reduction, so a hazard ratio of point seven five, would have ninety percent 00:08:00.843 --> 00:08:03.993 power of, to detect a difference. 00:08:03.993 --> 00:08:05.913 They're gonna do a log-rank test. 00:08:06.133 --> 00:08:09.423 They're gonna look at the hazard ratio in a Cox model 00:08:09.743 --> 00:08:15.483 for, um, the, the cardiovascular events, time to first event. 00:08:16.243 --> 00:08:19.873 Now it does-- And then it says this, the nineteen hundred is a 00:08:19.873 --> 00:08:23.433 bit, they're gonna enroll that, but they're gonna wait for the events. 00:08:23.623 --> 00:08:29.153 It's an event-driven trial, five hundred and fifty primary, uh, outcome 00:08:29.183 --> 00:08:31.993 events, assuming a log-rank test. 00:08:33.703 --> 00:08:39.623 Uh, ninety percent power for a point seven five hazard ratio. 00:08:40.003 --> 00:08:43.583 And it also says the number of events provides eighty percent 00:08:43.583 --> 00:08:47.253 pow-power for a twenty-two percent risk reduction, point seven eight. 00:08:48.013 --> 00:08:52.583 It's, we'll come back to that interesting point seven eight has 00:08:52.633 --> 00:08:55.713 eighty percent power, uh, in the trial. 00:08:56.813 --> 00:09:01.713 Now, the first thing I, I try to diagnose from the trial was this 00:09:01.713 --> 00:09:06.673 adaptive i-in any way in the trial, or is it a trial and we're gonna 00:09:06.673 --> 00:09:08.503 wait five years and look at the data? 00:09:09.033 --> 00:09:15.413 Well, it says two formal interim analyses are planned to assess efficacy 00:09:15.643 --> 00:09:20.533 when approximately one-third, which is pretty early in event-driven trials, 00:09:20.563 --> 00:09:27.311 and two-thirds of the primary endpoint outcomes have been observed They 00:09:27.321 --> 00:09:30.221 use Hay-Bittle-Peto stopping rules. 00:09:30.231 --> 00:09:34.281 It says, uh, using three standard deviations at either 00:09:34.281 --> 00:09:35.481 of the first two interims. 00:09:35.961 --> 00:09:42.441 So that's a, a, a nominal alpha of, of 0.27, 00:09:43.001 --> 00:09:45.851 um, uh, within that. 00:09:46.261 --> 00:09:52.671 So the, uh, you know, the typical two point five, this is, this is 0.27, 00:09:53.321 --> 00:09:57.991 uh, which has it, it describes minimal impact on the nominal type one 00:09:57.991 --> 00:09:59.681 error rate by the end of the trial. 00:09:59.681 --> 00:10:03.801 So four point eight two percent is the alpha. 00:10:04.191 --> 00:10:08.791 So, uh, uh, at the end of the trial, so it pre- preserves a good bit of it. 00:10:09.741 --> 00:10:12.901 Now, it says nothing about futility analyses in here. 00:10:12.901 --> 00:10:18.121 It says a detailed DSMB charter will be developed before starting the study 00:10:18.121 --> 00:10:20.371 in consultation with DSMB members. 00:10:20.371 --> 00:10:21.591 That's essentially the extent of it. 00:10:21.881 --> 00:10:26.541 I looked through the SAP, uh, that which was a, uh, which was an 00:10:26.721 --> 00:10:29.001 appendix to this, a supplement to it. 00:10:29.301 --> 00:10:32.161 Looked through the SAP, and this is really the wording it gives there. 00:10:32.171 --> 00:10:37.171 I think in hindsight, the answer is there were no futility analyses in the trial. 00:10:38.151 --> 00:10:44.321 So it's a superiority trial for rivaroxaban on cardiovascular events. 00:10:45.781 --> 00:10:48.101 Now, what happened in the trial? 00:10:49.141 --> 00:10:57.541 So it says, and this is in the, in the paper, in the report, on July 8th, 2025, 00:10:57.541 --> 00:11:03.601 during a pre-specified review of interim data, and it's kind of interesting 00:11:03.601 --> 00:11:09.261 because the, the-- it says two hundred and fifty-four primary outcome events, which 00:11:09.261 --> 00:11:12.961 is fifty percent of the planned events. 00:11:14.111 --> 00:11:16.901 Fourteen hundred and thirty-two patients randomized. 00:11:16.941 --> 00:11:20.321 Remember they said nineteen hundred, so fourteen hundred and thirty-two 00:11:20.321 --> 00:11:24.131 randomized, and it, it's at two hundred and fifty-four events. 00:11:24.131 --> 00:11:27.461 They said that they're gonna do a superiority at a third of 00:11:27.541 --> 00:11:29.391 events and two-thirds of events. 00:11:29.761 --> 00:11:36.231 So it says on July 8th, uh, fifty percent of the events, two hundred and fifty-four. 00:11:37.731 --> 00:11:43.771 The Data and Safety Monitoring Board recommended early termination of the trial 00:11:44.811 --> 00:11:52.391 to concerns regarding net harm and a low probability of demonstrating efficacy 00:11:54.743 --> 00:11:59.843 Okay, so the DSMB made this recommendation, and then it says 00:11:59.853 --> 00:12:05.483 the recommendation was not based on pre-specified stopping rules. 00:12:06.443 --> 00:12:11.293 So from all my read, there were no pre-specified futility, but the 00:12:11.293 --> 00:12:14.503 DSMB makes this recommendation. 00:12:15.373 --> 00:12:19.133 Net harm and low probability of demonstrating efficacy. 00:12:19.563 --> 00:12:21.933 And then it says it gives additional detail. 00:12:21.953 --> 00:12:28.993 It says, "Rather, the decision was based on a post hoc analysis 00:12:29.043 --> 00:12:38.133 estimating a conditional power of 16% for a hazard ratio of 0.78, 00:12:40.343 --> 00:12:45.723 indicating a low likelihood of demonstrating efficacy even if 00:12:45.723 --> 00:12:52.303 the trial had continued until the planned 515 primary outcome events." 00:12:54.029 --> 00:12:57.539 Okay, so a lot to unpack here. 00:12:58.369 --> 00:13:00.929 So no pre-specified rules. 00:13:00.949 --> 00:13:02.049 They do a post-hoc. 00:13:02.049 --> 00:13:05.439 I think with no pre-specified rules, everything is post-hoc. 00:13:05.449 --> 00:13:10.519 So I, I, I found that wording somewhat weird, uh, within that setting 00:13:10.889 --> 00:13:13.169 as though that's super relevant. 00:13:13.249 --> 00:13:13.589 But… 00:13:14.309 --> 00:13:19.139 And they do conditional power, and they say a low likelihood 00:13:19.139 --> 00:13:20.399 of demonstrating efficacy. 00:13:20.409 --> 00:13:22.839 So first of all, what is conditional power? 00:13:22.969 --> 00:13:31.329 16% probability of demonstrating s- uh, superiority at the final 00:13:31.329 --> 00:13:33.329 analysis, and it says 16%. 00:13:34.329 --> 00:13:41.829 Now, conditional power is given the current data at the interim at 254 events. 00:13:42.409 --> 00:13:48.649 It's a calculation of what's the probability of seeing superiority 00:13:48.649 --> 00:13:49.829 at the end of the trial. 00:13:51.389 --> 00:13:57.619 Now, conditional power assumes a single value for the hazard ratio, 00:13:58.329 --> 00:14:00.529 and it can be done in many ways. 00:14:00.989 --> 00:14:06.489 Sometimes conditional power looks at the current estimate of the hazard ratio. 00:14:07.439 --> 00:14:13.709 Sometimes it uses a preset hazard ratio like it did here. 00:14:14.439 --> 00:14:18.699 So it assumes a hazard ratio of 0.78 00:14:18.699 --> 00:14:23.959 for the rest of the trial, uh, with variability of the rest of the data. 00:14:23.989 --> 00:14:26.449 Suppose the truth is 0.78. 00:14:27.979 --> 00:14:34.629 What's the probability that at 550-- 15 events we see superiority? 00:14:34.849 --> 00:14:39.769 That's the calculation that's made, and that's 16%. 00:14:40.969 --> 00:14:43.739 My first reaction is that's not that low. 00:14:44.539 --> 00:14:49.439 That's, that, that, that likelihood, I d- I don't find that low at all. 00:14:50.249 --> 00:14:52.109 Uh, it was interesting. 00:14:52.119 --> 00:14:58.289 My, my son, Cooper, who I've talked about on this show before, he's, um, 00:14:58.679 --> 00:15:05.559 he was in seventh grade, and we h- would not let him play tackle football. 00:15:06.819 --> 00:15:13.369 And he came to us with an argument that he should be allowed to play tackle football. 00:15:14.559 --> 00:15:23.539 And as part of his argument, he found, uh, uh, data that said the 00:15:23.539 --> 00:15:34.425 likelihood of a neurological event For a kid playing tackle football, 00:15:34.505 --> 00:15:38.255 a serious injury was only 13%. 00:15:39.635 --> 00:15:43.485 And he presented that in his PowerPoint presentation to us. 00:15:43.715 --> 00:15:50.955 Yes, we have a weird family, um, in this which, uh, that was only 13%, so he 00:15:50.955 --> 00:15:54.325 should be allowed to play tackle football. 00:15:55.545 --> 00:16:00.365 And our reaction was, "Oh my God, 13% is enormous." 00:16:00.685 --> 00:16:08.025 You know, the 13% chance of, of serious injury in that setting is very large. 00:16:08.145 --> 00:16:13.015 So 16% probability of demonstrating superiority by the end of the 00:16:13.015 --> 00:16:17.385 trial doesn't strike me as that low, actually, that unlikely. 00:16:18.805 --> 00:16:21.515 Now, there's multiple parts to this. 00:16:21.925 --> 00:16:24.935 It uses this .78 00:16:25.605 --> 00:16:27.415 to make this calculation. 00:16:29.175 --> 00:16:36.005 Now, I went back and figured out if the, uh, you can find the, the formula 00:16:36.015 --> 00:16:38.255 for conditional power, uh, in this. 00:16:38.255 --> 00:16:43.155 There's a beautiful normal approximation to the log hazard ratio. 00:16:43.715 --> 00:16:46.175 If you assume .78 00:16:46.475 --> 00:16:53.695 and the conditional power for 16% by the end of the trial, at the time, it 00:16:53.705 --> 00:16:55.995 means that the hazard ratio was 1.03. 00:16:55.995 --> 00:16:56.395 So it was worse than one at the 00:17:02.155 --> 00:17:07.755 time, and the, the data was doing worse. 00:17:08.745 --> 00:17:15.535 The 95% credible interval for that 1.03 00:17:15.545 --> 00:17:21.695 hazard ratio at the time of 254 events goes down to about .81. 00:17:24.335 --> 00:17:30.615 So somebody's calculating this conditional power for what's the chance this trial 00:17:30.615 --> 00:17:34.825 is successful, and they're using .78 00:17:38.021 --> 00:17:42.441 The 95% confidence interval goes down to 0.81 00:17:42.441 --> 00:17:44.421 as the lower bound of that. 00:17:44.441 --> 00:17:45.491 So 0.78 00:17:45.491 --> 00:17:50.881 is not even in the 95% confidence interval of when they're making this calculation. 00:17:50.881 --> 00:17:54.161 It's a very, very unlikely value. 00:17:55.281 --> 00:17:58.901 So what is the relevance of this calculation? 00:17:58.911 --> 00:18:06.651 It's a super odd calculation, and I think it's, it provides no information 00:18:06.661 --> 00:18:08.521 for the DSMB to make the decision. 00:18:08.681 --> 00:18:10.051 It's not a low number. 00:18:10.431 --> 00:18:15.941 You sh- probably shouldn't be stopping trials for futility at 16%. 00:18:16.031 --> 00:18:18.221 It's not that unusual of a number. 00:18:19.361 --> 00:18:24.201 But it's calculated using-- Remember when we talked about the way they 00:18:24.211 --> 00:18:30.931 powered the trial, they said it has 80% power, assuming 0.78. 00:18:30.931 --> 00:18:34.501 So halfway through, through the trial, if you still assume that 0.78, 00:18:34.501 --> 00:18:36.751 it's gone from 80% to 16%. 00:18:37.911 --> 00:18:41.781 Doesn't seem that super odd at the time. 00:18:42.031 --> 00:18:46.911 I, I-- That I feel like it's not a very informative number at all. 00:18:47.551 --> 00:18:52.101 You won't be surprised to find out I'm not really a fan of conditional power. 00:18:53.421 --> 00:18:57.141 Now, conditional power assumes you know the answer and what's the 00:18:57.141 --> 00:18:59.551 probability that that, that happens. 00:19:00.501 --> 00:19:05.331 And it uses a number that's incredibly unlikely given the data we've observed. 00:19:05.341 --> 00:19:09.921 And so it's, it's, it's a bad analysis in that. 00:19:09.951 --> 00:19:11.111 Now, what would I do? 00:19:11.521 --> 00:19:14.241 I would do a Bayesian predictive probability. 00:19:14.821 --> 00:19:20.871 I would use the current posterior distribution of given 254 events 00:19:20.881 --> 00:19:22.891 with a current hazard ratio of 1.03 00:19:25.031 --> 00:19:28.721 What is the posterior distribution of the hazard ratio? 00:19:29.661 --> 00:19:34.981 And then given that estimate of the hazard ratio, what's the probability the 00:19:34.981 --> 00:19:38.011 trial's going to demonstrate superiority? 00:19:39.471 --> 00:19:43.861 Okay, I did that with this data, and it comes out to be about 1.4 00:19:46.565 --> 00:19:54.485 So given the current data, the, and the current posterior distribution 00:19:54.485 --> 00:20:00.195 of the hazard ratio at the interim, integrating over that, the probability 00:20:00.195 --> 00:20:04.895 of seeing statistical significance at the end of the trial is 1.4%. 00:20:06.065 --> 00:20:08.795 Now, I think that's a really, really useful number. 00:20:08.795 --> 00:20:11.085 And by the way, it's a number we use. 00:20:11.595 --> 00:20:18.155 It's, it's a calculation we use very frequently in s- for stopping for futility 00:20:18.165 --> 00:20:24.385 because it incorporates the uncertainty at the interim, the variability of 00:20:24.385 --> 00:20:30.735 the data to come of demonstrating superiority or the goal of the trial. 00:20:30.785 --> 00:20:33.425 It's, it's a beautiful number. 00:20:33.865 --> 00:20:39.335 And you can use, you can use prior distributions for 00:20:39.335 --> 00:20:40.885 that that are informative. 00:20:41.395 --> 00:20:44.175 You could use fairly weak priors. 00:20:44.485 --> 00:20:51.395 Uh, that, that, uh, analysis prior that goes into that, that's a word that the 00:20:51.425 --> 00:20:55.975 FDA draft guidance used, the analysis prior that goes in to calculate that 00:20:56.375 --> 00:20:59.745 could be based on, uh, an optimist's view. 00:20:59.765 --> 00:21:04.465 And when an optimist thinks it's unlikely, you could stop the trial. 00:21:04.525 --> 00:21:08.815 Now, if your optimist believes 0.78 00:21:08.835 --> 00:21:12.135 with no variability, that's the conditional power. 00:21:12.585 --> 00:21:14.995 That's equivalent to what they calculated. 00:21:15.375 --> 00:21:19.405 You have a prior probability that the hazard ratio is 0.78 00:21:19.405 --> 00:21:20.765 with zero variability. 00:21:21.495 --> 00:21:24.165 You think the chance of winning this trial is 16%. 00:21:25.955 --> 00:21:30.975 I don't think that Optimus would stop the trial, but I think it's, it, it's not the 00:21:30.975 --> 00:21:33.905 right calculation at that particular time. 00:21:35.135 --> 00:21:38.245 Okay, so a lot went into that. 00:21:38.815 --> 00:21:46.335 Now, the other component to it is that at the time, they saw increased major 00:21:46.335 --> 00:21:52.185 bleeding, which was the safety endpoint, which is what anticoagulations do. 00:21:52.255 --> 00:21:56.925 They n- you know the bleeding's gonna be higher, but you're hoping it's offset 00:21:56.935 --> 00:22:00.405 by reduction in cardiovascular events. 00:22:00.725 --> 00:22:05.255 Now, the final data, I don't know what the data were at the interim on bleeding. 00:22:05.525 --> 00:22:13.145 The final data on bleeding was for the rivaroxaban arm, it was 8.8% 00:22:13.195 --> 00:22:14.185 major bleed. 00:22:15.155 --> 00:22:22.225 That was 64 events, uh, on that arm, 44 events on placebo for a 00:22:22.225 --> 00:22:25.375 s- a 6% rate of major bleeding. 00:22:25.775 --> 00:22:28.795 So that went from six to 8.8, 00:22:29.245 --> 00:22:35.965 which surely went into the DSMB's decision to recommend stopping the trial. 00:22:36.785 --> 00:22:42.625 The probability of showing superiority, and I think they used an awkward number 00:22:42.625 --> 00:22:45.495 for that, but with this elevated rate. 00:22:45.495 --> 00:22:47.795 Now, we know major bleeding's gonna be higher. 00:22:47.805 --> 00:22:49.275 That's what the drug does. 00:22:49.655 --> 00:22:52.335 But the perception of this 2.8% 00:22:52.345 --> 00:22:56.405 increase with a hazard ratio of 1.3 00:22:56.405 --> 00:23:01.345 at the time had them recommend stopping for futility. 00:23:02.775 --> 00:23:08.735 Now, the final result of the trial was a hazard ratio on 00:23:08.765 --> 00:23:11.165 cardiovascular events that was 1.09. 00:23:11.435 --> 00:23:13.335 There was additional follow-up. 00:23:13.675 --> 00:23:19.105 They, they, um, at this July event, they stopped the trial a month 00:23:19.105 --> 00:23:22.425 later, so there were deliberations on this, but they stopped it. 00:23:22.425 --> 00:23:25.425 There was additional follow-up, probably additional events 00:23:25.435 --> 00:23:26.735 going through adjudication. 00:23:27.015 --> 00:23:31.295 With the final events, it ended with a hazard ratio of 1.09 00:23:31.615 --> 00:23:40.235 in that, um, 13, uh, uh, events per 100 years on rivaroxaban, 11.8 00:23:40.235 --> 00:23:41.075 on placebo. 00:23:41.635 --> 00:23:47.285 It was doing worse than placebo, uh, at this particular time, and stopped 00:23:47.285 --> 00:23:54.445 with 1,463 patients randomized, about five- 400 and some plus patients less 00:23:54.745 --> 00:23:57.925 than what they planned to enroll in it. 00:23:58.955 --> 00:23:59.545 Okay. 00:24:00.135 --> 00:24:04.285 So overall, what does this mean? 00:24:04.935 --> 00:24:08.315 So I think the DSMB got it right. 00:24:09.005 --> 00:24:14.855 Um, knowing what I know of the results, uh, I would have recommended stopping. 00:24:15.595 --> 00:24:21.055 But I think the problem is, I don't think that's the group to make this decision 00:24:22.497 --> 00:24:27.477 Uh, the thing that bothers me here is there were no prospective futility 00:24:27.477 --> 00:24:32.433 rules So now you turn over this trial. 00:24:33.113 --> 00:24:39.413 It is an incredible amount of effort by a ton of people, 1400 patients agreeing 00:24:39.413 --> 00:24:44.913 to enter this trial, and there's no rules for stopping the trial for futility. 00:24:44.923 --> 00:24:48.173 You leave it up to smart people on the DSMB. 00:24:48.963 --> 00:24:52.533 But those are really hard decisions for a DSMB. 00:24:52.723 --> 00:24:57.123 I think this is something that the PIs in the trial should define. 00:24:57.513 --> 00:25:01.323 They should shet- set up what, what is futility? 00:25:01.633 --> 00:25:06.553 What is the be- should it be a balance of, of bleeding to, to that? 00:25:06.913 --> 00:25:10.313 Are they happy that the trial was stopped at this particular time? 00:25:10.553 --> 00:25:13.423 Took about a month before they, they eventually stopped it. 00:25:13.443 --> 00:25:14.613 Did they review the data? 00:25:14.643 --> 00:25:15.683 Did they agree with this? 00:25:16.783 --> 00:25:19.833 Uh, I feel like it has to be part of the design. 00:25:20.233 --> 00:25:24.363 It's frustrating as a statistician to read trials like this that 00:25:24.363 --> 00:25:26.123 don't have futility rules set up. 00:25:26.473 --> 00:25:29.403 It's a five-year trial with nineteen hundred patients. 00:25:30.423 --> 00:25:34.313 Cardiovascular death is part of this endpoint, major bleeding. 00:25:35.153 --> 00:25:40.903 So I think the DSMB did right, but they're in a really hard spot 00:25:40.913 --> 00:25:42.613 to make a decision for futility. 00:25:42.843 --> 00:25:44.453 Is sixteen percent right? 00:25:44.473 --> 00:25:47.853 Is that conditional power number the right thing in it? 00:25:48.333 --> 00:25:54.573 Now, this is a trial looking at a readily available treatment, so this is kind of 00:25:54.963 --> 00:25:56.983 comparative effectiveness, if you will. 00:25:56.983 --> 00:26:02.053 I don't think this is gonna result-- Maybe it results in some level of, 00:26:02.163 --> 00:26:09.353 of guideline saying, "Oh, patients with CKD stage four or five should 00:26:09.353 --> 00:26:13.763 take low-dose rivaroxaban," if the data were in a particular situation. 00:26:13.763 --> 00:26:15.313 But it's not regulatory approval. 00:26:15.313 --> 00:26:17.573 It's not a sponsor going after this. 00:26:17.573 --> 00:26:20.153 I don't, I don't believe, uh, in the setting. 00:26:21.053 --> 00:26:26.783 Um, but especially in sponsor-designed trials, the sponsor should make 00:26:26.793 --> 00:26:28.543 the decision about futility. 00:26:28.853 --> 00:26:30.663 When is enough is enough? 00:26:30.703 --> 00:26:32.573 When should we stop the trial? 00:26:32.873 --> 00:26:34.433 It's a business decision. 00:26:34.433 --> 00:26:35.793 It's a money decision. 00:26:35.823 --> 00:26:39.683 There's an ethical component to it about asking patients to be 00:26:39.683 --> 00:26:46.113 randomized into a trial with a very low likelihood of demonstrating a, 00:26:46.113 --> 00:26:48.373 a, a result that changes practice. 00:26:48.373 --> 00:26:52.573 And, uh, so are they contributing to science if the probability of success 00:26:52.573 --> 00:26:54.123 is one point four four percent? 00:26:54.723 --> 00:26:59.423 All these questions should go into the design of the trial, not 00:26:59.803 --> 00:27:04.853 humans that have to sit around and make this decision post-hoc in it. 00:27:05.593 --> 00:27:08.603 The last part about it is I, I think conditional power 00:27:08.613 --> 00:27:10.313 is just a bad tool for that. 00:27:10.823 --> 00:27:12.193 It's a bad tool. 00:27:12.593 --> 00:27:16.703 Integrate over the uncertainty of what you know at the time, reflective that point 00:27:16.703 --> 00:27:21.623 seven eight was very, very unlikely, so sixteen percent is just the wrong measure. 00:27:21.823 --> 00:27:25.893 Whether it's big or small, it- it's the wrong measure So I'd love to 00:27:25.903 --> 00:27:33.243 see them use a much more realistic, relevant calculation in that. 00:27:34.883 --> 00:27:41.573 Okay, so that was article number one that shows up in my email box. 00:27:42.393 --> 00:27:47.393 Uh, article number two is the VICTORY randomized clinical trial. 00:27:47.443 --> 00:27:48.483 Love the names. 00:27:49.233 --> 00:27:51.943 And this one caught my attention right off the bat. 00:27:51.943 --> 00:27:53.413 This is the first one I looked at. 00:27:53.413 --> 00:28:01.153 It was-- This came out June 10th, published online in JAMA, June 10th, 2026. 00:28:01.153 --> 00:28:08.403 And this is the VICTORY trial, and it is high-dose intravenous vitamin 00:28:08.403 --> 00:28:17.513 C, uh, studying mortality and organ dysfunction in severe burn injury. 00:28:18.323 --> 00:28:19.773 I don't, I don't know much. 00:28:19.773 --> 00:28:23.303 We've done a little bit in burn, but I don't, I don't know much about it. 00:28:23.683 --> 00:28:30.113 Um, uh, the d- the, the syndrome, uh, within it, you can imagine 00:28:30.123 --> 00:28:31.703 the clinical syndrome of this. 00:28:32.103 --> 00:28:35.023 Uh, but I don't know much about treatment in this or endpoints. 00:28:35.093 --> 00:28:39.963 So, but it struck me was the vitamin C part of it. 00:28:40.563 --> 00:28:43.243 Why did vitamin C strike me? 00:28:43.243 --> 00:28:53.273 Well, um, in, in our REMAP-CAP trial, we investigated, um, REMAP-CAP and LOVIT. 00:28:54.423 --> 00:28:59.883 LOVIT was a trial investigating vitamin C in sepsis at the time, 00:29:00.353 --> 00:29:03.903 and COVID, the pandemic broke out. 00:29:04.203 --> 00:29:08.923 The two trials combined their data together to investigate and 00:29:08.923 --> 00:29:17.243 randomize patients with COVID and investigating does high-dose vitamin 00:29:17.243 --> 00:29:23.993 C improve outcomes in severe COVID 00:29:26.029 --> 00:29:30.549 And our trial came out with-- and the endpoint was organ 00:29:30.549 --> 00:29:33.859 support-free days, in the back of my mind, I remember this result. 00:29:34.119 --> 00:29:40.999 It came out, it estimated essentially harm on the ordinal endpoint of mortality 00:29:41.029 --> 00:29:47.589 and then organ support-free days, it came out with a adjusted odds ratio of .88, 00:29:47.849 --> 00:29:54.679 less than one is harm, with a ninety-one point four percent probability of harm. 00:29:55.279 --> 00:30:04.359 It was doing worse on mortality, uh, so on survival was fifty-seven percent in 00:30:04.369 --> 00:30:09.229 the vitamin C group and sixty percent on the non-vitamin C, the control. 00:30:09.709 --> 00:30:10.899 So it did worse. 00:30:10.909 --> 00:30:16.699 The trial was stopped for futility, uh, uh, in, uh, in that case. 00:30:16.709 --> 00:30:20.199 So I knew this vitamin C did not do well in COVID. 00:30:21.059 --> 00:30:25.279 And then I saw that LOVEIT read out in sepsis. 00:30:25.859 --> 00:30:31.579 And so this, the paper came out, um, uh, came out in two 00:30:31.579 --> 00:30:38.009 thousand twenty-two reporting the results of vitamin C compared to 00:30:38.009 --> 00:30:41.829 placebo in patients with sepsis. 00:30:42.889 --> 00:30:50.819 And the, um, the primary endpoint in that trial was persistent organ 00:30:50.819 --> 00:30:53.799 dysfunction, uh, or mortality. 00:30:54.599 --> 00:31:00.639 So they died or I believe at day twenty-eight they still had 00:31:01.259 --> 00:31:03.039 persistent organ dysfunction. 00:31:03.039 --> 00:31:06.339 They couldn't get off organ support, a bad outcome. 00:31:06.409 --> 00:31:09.929 It was a dichotomous outcome of that. 00:31:10.449 --> 00:31:14.959 And in the vitamin C group, the proportion of patients that met that 00:31:14.959 --> 00:31:19.329 condition, died or organ dysfunction was forty-four point five percent. 00:31:19.919 --> 00:31:23.359 In placebo, it was thirty-eight point five percent. 00:31:23.409 --> 00:31:30.819 So a six percent increase in that primary outcome in sepsis, uh, statistically 00:31:30.819 --> 00:31:35.169 significant, uh, harm in that trial. 00:31:35.179 --> 00:31:39.689 So it sort of-- I, I remembered vitamin C was not doing well, 00:31:40.059 --> 00:31:42.769 uh, in, in those two trials. 00:31:42.799 --> 00:31:48.129 So here vitamin C is being given to patients with severe burn injury 00:31:50.309 --> 00:31:52.849 So I said, "Oh, I've, I've got to look at, see this. 00:31:52.849 --> 00:31:54.629 Does vitamin C work here?" 00:31:55.099 --> 00:31:59.889 So the trial design, one-to-one randomized high-dose vitamin 00:31:59.889 --> 00:32:02.679 C given by IV versus placebo. 00:32:03.269 --> 00:32:09.539 The primary outcome was a composite of 28-day mortality, so clearly severe 00:32:09.549 --> 00:32:17.249 burn, and persistent organ dysfunction, uh, defined as dependence on mechanical 00:32:17.249 --> 00:32:22.829 ventilation, kidney replacement, or needing vasopressors inotrope. 00:32:22.839 --> 00:32:27.079 So, uh, still, uh, needing, you know, low blood pressure vasopressors. 00:32:28.349 --> 00:32:31.009 Uh, you'd have that support at 28 days. 00:32:31.219 --> 00:32:37.079 So this, uh, this composite outcome of bad things at day 28 is the 00:32:37.079 --> 00:32:42.609 primary endpoint, similar to the LOVIT trial actually, for the Burn trial. 00:32:43.479 --> 00:32:50.219 So the total sample size was planned three-333 patients per group. 00:32:50.649 --> 00:32:59.139 So 666-patient trial, which would provide 78% power, uh, for 00:32:59.139 --> 00:33:07.809 demonstrating a decrease in that outcome at 27% for placebo to 18%. 00:33:08.949 --> 00:33:13.839 So that, that reduction, it would be 78% powered. 00:33:14.089 --> 00:33:19.029 So still relatively rare, 27% on placebo was thought to 00:33:19.029 --> 00:33:20.289 be the rate of this outcome. 00:33:21.119 --> 00:33:24.609 My first thought as a statistician, I'm just jumping in, is when I 00:33:24.609 --> 00:33:27.959 read this is, boy, there's got to be a better endpoint here. 00:33:28.359 --> 00:33:32.359 A dichotomous outcome of these bad events, but there's got to be more 00:33:32.399 --> 00:33:37.449 to this to, to look at this outcome where you learn something from the 00:33:37.459 --> 00:33:43.189 73% of patients that aren't dead or aren't in this really bad state. 00:33:43.489 --> 00:33:49.349 Now, death is worse than being on vasopressors on day 28 and, and likewise. 00:33:49.359 --> 00:33:51.419 So I don't like the endpoint. 00:33:51.419 --> 00:33:52.879 I'd like to see something better. 00:33:53.619 --> 00:33:59.079 Now, that isn't so much the point of this, but as a statistician reads through 00:33:59.079 --> 00:34:01.749 this, I cringe a little bit at that. 00:34:02.319 --> 00:34:08.789 Now, it says, while allowing for two interim analyses for futility/safety 00:34:11.757 --> 00:34:16.317 The, and it says, "The operating characteristics of the trial under 00:34:16.317 --> 00:34:18.787 the pre-specified futility rule." 00:34:19.127 --> 00:34:24.517 So they have a pre-specified futility rule here in this vitamin C may 00:34:24.517 --> 00:34:29.557 be triggered by the poor results of these other trials, the harm, 00:34:30.247 --> 00:34:32.657 the reasonable likelihood of harm. 00:34:33.037 --> 00:34:37.707 In one case, 92%, the other one was statistically significantly harmful. 00:34:38.517 --> 00:34:46.717 So the pre-specified futility rule is an adjusted risk ratio of greater than 1.1. 00:34:46.727 --> 00:34:50.807 So less than one is good, favors vitamin C. 00:34:51.667 --> 00:35:01.087 So at these two pre-specified interims, if they saw a relative risk above 1.1 00:35:01.087 --> 00:35:05.857 favoring control, it-- they would s- they would recommend futility. 00:35:05.867 --> 00:35:10.667 That was the pre-specified rule, and these w- these interims were to take 00:35:10.787 --> 00:35:16.587 place at one-third and two-third of the events that if it was above 1.1, 00:35:16.587 --> 00:35:18.457 they would stop for futility. 00:35:19.097 --> 00:35:22.447 And it says it was determined through simulation. 00:35:23.117 --> 00:35:27.777 So in the SAP, they actually go through and they look at the operating 00:35:27.777 --> 00:35:33.507 characteristics of this futility rule, and they talk about, uh, 00:35:33.547 --> 00:35:37.327 it reduces power from 79 to 78%. 00:35:37.327 --> 00:35:42.167 Futility rules reduce power because every once in a while, the trial 00:35:42.327 --> 00:35:46.097 w- hits those futility rules, depending on what it is, and that 00:35:46.097 --> 00:35:47.957 would've gone on to be successful. 00:35:48.157 --> 00:35:52.147 Depending on that rule, you can see reduction in power. 00:35:52.147 --> 00:35:56.107 So they, they report what it is, and it says minimal effect on bias, 00:35:56.107 --> 00:36:01.567 which, um, uh, futility rules, uh, any stopping rule has bias. 00:36:01.567 --> 00:36:05.297 Bias is not bad, and in this case, this bias is not bad. 00:36:05.297 --> 00:36:07.287 So they say it's minimal, so that's good. 00:36:07.317 --> 00:36:12.647 But they investigated, they created a pre-specified futility rule in this 00:36:12.657 --> 00:36:18.167 trial as opposed to the track trial where there were no, uh, futility rules. 00:36:18.697 --> 00:36:24.397 Now, it doesn't mention anything about efficacy, so it looks like they did two 00:36:24.397 --> 00:36:27.277 futility rules and, and not for efficacy. 00:36:27.617 --> 00:36:30.047 Fantastic, uh, in the trial. 00:36:30.667 --> 00:36:35.047 Now, I might throw out a little bit of, you know, superiority at the 00:36:35.047 --> 00:36:39.247 time, but, uh, maybe they wanted enough data to demonstrate and, and, 00:36:39.247 --> 00:36:42.951 and change practice in that Okay. 00:36:44.001 --> 00:36:52.011 So two interim analyses were pre-planned at 222 patients, 444 patients of the 00:36:52.761 --> 00:36:56.551 666 total sample size, uh, in it. 00:36:57.001 --> 00:37:01.161 Uh, by the way, I wonder, again, statistician reading this, they 00:37:01.161 --> 00:37:08.131 report the sample size in the trial as 333 per group, and then 00:37:08.141 --> 00:37:14.031 later they talk about the interims happening at 222 total and 444 total. 00:37:14.531 --> 00:37:18.911 Did they not wanna write the sample size of 666? 00:37:20.021 --> 00:37:21.851 I throw that out there as a possibility. 00:37:21.861 --> 00:37:24.471 So they re- said it was 333 per group. 00:37:25.501 --> 00:37:28.241 Um, is 1.1, 00:37:28.241 --> 00:37:30.551 and so what happened in the trial? 00:37:30.991 --> 00:37:32.791 This is where they report that. 00:37:33.121 --> 00:37:40.431 This threshold was crossed at the first interim analysis, prompting the DSMB 00:37:40.431 --> 00:37:42.971 to recommend termination of the trial. 00:37:43.261 --> 00:37:46.321 By the way, I think that's a, a very reasonable thing to do, that you have 00:37:46.351 --> 00:37:52.091 pre-specified rules, but you have a human group that's looking at the data and they 00:37:52.091 --> 00:37:53.951 say, "Yes, we think it's appropriate." 00:37:54.051 --> 00:38:01.249 But you've predefine what that is in the setting So predetermined 00:38:01.249 --> 00:38:03.189 rules hit the trigger. 00:38:03.429 --> 00:38:07.829 DSMB said, "Yes, you should stop the trial," and the trial 00:38:07.839 --> 00:38:10.839 was stopped in this setting. 00:38:11.599 --> 00:38:14.489 Okay, what were the data? 00:38:15.339 --> 00:38:21.899 The data were on this endpoint were, it was at 120 patients 00:38:21.899 --> 00:38:23.719 randomized on vitamin C, 00:38:26.749 --> 00:38:26.759 41%. 00:38:26.759 --> 00:38:30.919 Remember they talked about making 27% on control go to 18. 00:38:32.229 --> 00:38:32.789 There were… 00:38:32.989 --> 00:38:34.969 I, I, I'll do placebo first. 00:38:35.279 --> 00:38:41.209 30% of placebo patients met this endpoint, very close to the 27 that 00:38:41.209 --> 00:38:44.409 they said, which was a, a pretty good design estimate of that. 00:38:44.769 --> 00:38:51.359 It was 41% on vitamin C, an increase from 30 to 41%. 00:38:52.439 --> 00:38:56.899 The Fisher exact test p-value at the time was .08, 00:38:57.129 --> 00:39:01.839 of course, on the way to harm, um, uh, in that setting. 00:39:02.159 --> 00:39:08.759 Uh, I don't know why you give two-sided p-values in a superiority trial, but 00:39:09.449 --> 00:39:15.029 we, we, we, we can interpret that, uh, Fisher, Fisher exact p-value, of course, 00:39:15.059 --> 00:39:17.469 going the wrong way in the setting. 00:39:17.809 --> 00:39:24.079 The mortality rate on placebo was 8%, so 9 of 118. 00:39:24.449 --> 00:39:29.479 It was 15%, 18 out of 120 on… 00:39:29.549 --> 00:39:31.719 I'm sorry, I, I may have said that wrong. 00:39:31.729 --> 00:39:35.959 On placebo it was 8%, nine of 118. 00:39:36.929 --> 00:39:43.749 On vitamin C it was 15%, so it went from 8 to 15%, almost a doubling of the 00:39:43.749 --> 00:39:45.929 mortality rate, uh, within the setting. 00:39:46.519 --> 00:39:52.789 And it hit the, the relative risk, the final relative risk was 1.38, 00:39:53.749 --> 00:39:56.109 which hit the 1.1 00:39:56.109 --> 00:40:00.689 and the recommendation, recommendation to stop, uh, for futility 00:40:02.733 --> 00:40:03.293 Okay. 00:40:03.383 --> 00:40:09.213 So overall within this, uh, and by the way, the, the result, I, 00:40:09.263 --> 00:40:11.263 I, I love reading the conclusion. 00:40:11.263 --> 00:40:19.083 So I imagine, uh, clinicians don't have time to read all these, these results, 00:40:19.083 --> 00:40:21.193 but they wanna get the highlights of this. 00:40:21.213 --> 00:40:24.623 And that's why I think this little cartoon they do in JAMA, and I think 00:40:24.623 --> 00:40:28.203 New England and other journals do that, is a summary of the trial results. 00:40:28.893 --> 00:40:32.683 And many times I disagree with the conclusion. 00:40:32.683 --> 00:40:34.353 I strikingly disagree with it. 00:40:34.353 --> 00:40:35.953 I think it's, it's bad. 00:40:37.013 --> 00:40:44.073 But here it says the conclusion is high-dose intravenous vitamin C did not 00:40:44.073 --> 00:40:47.763 reduce mortality or organ dysfunction. 00:40:49.283 --> 00:40:53.983 So again, that's, it, you know, didn't reach statistical significance 00:40:54.973 --> 00:41:00.603 and may be associated with harm in patients with severe burn injury. 00:41:01.143 --> 00:41:04.493 I think that's pretty reasonable in this setting. 00:41:04.493 --> 00:41:07.313 It wasn't statistically significantly harmful. 00:41:07.623 --> 00:41:11.263 You know, this P value, if we, if we do a Bayesian interpretation 00:41:11.263 --> 00:41:15.023 of the P value, there's a ninety-six percent chance of harm. 00:41:15.543 --> 00:41:19.993 So I, I would've been upset had they just said, "Did not reduce mortality 00:41:19.993 --> 00:41:21.943 of an or- or organ dysfunction." 00:41:22.243 --> 00:41:25.843 I think there's reasonable evidence that it is harmful. 00:41:26.253 --> 00:41:31.343 And of course, I just in my results with vitamin C, vitamin C seems to be 00:41:31.493 --> 00:41:33.753 harmful for people with severe disease. 00:41:34.063 --> 00:41:39.163 This is a third result now where it does worse and seems to be harmful. 00:41:39.563 --> 00:41:41.943 Uh, so it says may be associated with harm. 00:41:41.943 --> 00:41:43.863 I thought very reasonable conclusion. 00:41:44.973 --> 00:41:45.373 Okay. 00:41:45.693 --> 00:41:47.373 So I liked this. 00:41:47.683 --> 00:41:50.893 I liked this, that they put this futility rule in there. 00:41:51.153 --> 00:41:56.653 By the way, I think it could have stopped earlier than that, um, uh, in the trial. 00:41:56.663 --> 00:42:00.123 I think had they seen it even earlier, they might have stopped for 00:42:00.123 --> 00:42:01.803 futility, but they had it set up. 00:42:02.423 --> 00:42:07.313 Now, I don't li-- a- and by the way, it stopped at thirty-three 00:42:07.313 --> 00:42:08.953 percent of the sample size. 00:42:09.463 --> 00:42:15.153 So it stopped at roughly two hundred and forty-ish patients rather than six 00:42:15.153 --> 00:42:20.283 hundred and sixty-six patients and likely doing harm to those patients, certainly 00:42:20.283 --> 00:42:27.413 not contributing to, uh, uh, scientific results that are gonna change care, uh, 00:42:27.413 --> 00:42:33.013 in this setting of that vitamin C is a good thing to add to the, the, the, the 00:42:33.023 --> 00:42:35.293 way to care for patients with severe burn. 00:42:35.683 --> 00:42:40.863 I don't love the rule of relative risk greater than one point one. 00:42:41.513 --> 00:42:46.663 At a third of the way through the trial, one point one is a very 00:42:46.663 --> 00:42:51.103 different predictive probability than two-thirds of the way of trial, a one 00:42:51.103 --> 00:42:58.459 point one And now this isn't where the, you know the number of events. 00:42:59.509 --> 00:43:03.549 Uh, this could have been an event-driven trial, but it's rarely done in sort 00:43:03.549 --> 00:43:09.249 of a binary yes, no at day 28 kind of thing, so they do number of patients. 00:43:09.739 --> 00:43:14.999 So had the events gone up and down, one point one is a very different statistical 00:43:14.999 --> 00:43:19.959 conclusion as to the likelihood of success of the trial if the event on 00:43:19.959 --> 00:43:22.909 control would have been 15% or 40%. 00:43:23.309 --> 00:43:26.379 It means very, very different things a third of the way through. 00:43:26.879 --> 00:43:30.809 Now, they simulated the rule and presumably under a number of 00:43:30.809 --> 00:43:35.919 different scenarios, different control rates, but I'd much prefer 00:43:36.989 --> 00:43:38.339 predictive probability in that. 00:43:38.369 --> 00:43:44.129 I think I'd even prefer conditional power, but using the MLE, I don't 00:43:44.129 --> 00:43:48.529 love it, and there's better ways to do it, but then just an absolute value. 00:43:48.699 --> 00:43:55.659 But when you do sufficient, um, simulations of that and you, you time 00:43:55.659 --> 00:44:00.689 it and carry out the analyses at the appropriate time, you know that, that's a, 00:44:00.749 --> 00:44:03.199 that's a reasonable summary of the data. 00:44:03.239 --> 00:44:06.449 Now here, you know, there could have been less events, and 00:44:06.449 --> 00:44:07.689 it's a different kind of thing. 00:44:07.689 --> 00:44:10.589 So I don't love the rule, but I love the fact that they did 00:44:10.589 --> 00:44:15.679 futility, was pre-specified, they carried out the futility analyses. 00:44:16.069 --> 00:44:20.339 Kind of a no-brainer that this trial, very low likelihood of 00:44:20.339 --> 00:44:22.209 success and high likelihood of harm. 00:44:22.599 --> 00:44:25.099 Uh, and by, by the way, should have calculated the Bayesian 00:44:25.099 --> 00:44:27.389 predictive probability at the time. 00:44:27.779 --> 00:44:30.399 Uh, much less than 1%. 00:44:30.829 --> 00:44:34.339 No chance of success in this trial, uh, on it. 00:44:34.369 --> 00:44:37.109 So I, I commend the designers on it. 00:44:37.149 --> 00:44:38.719 I would have done it differently. 00:44:39.439 --> 00:44:40.929 Lots of flavors to that. 00:44:42.169 --> 00:44:44.019 So two papers in JAMA. 00:44:44.069 --> 00:44:48.249 Interestingly, you know, a lot of people talk about these medical journals as, 00:44:48.519 --> 00:44:53.469 you know, they only publish positive results, publication bias in the setting. 00:44:53.739 --> 00:44:59.209 Here are two papers published where futility was the result of both, both 00:44:59.219 --> 00:45:01.429 trials, and they got published in JAMA. 00:45:01.429 --> 00:45:04.119 I think really important results I, I suspect. 00:45:04.589 --> 00:45:10.029 I don't know whether vitamin C is commonly done in burns, um, uh, in that, 00:45:10.039 --> 00:45:16.359 but I imagine anticoagulation is, is used and not that uncommon in patients 00:45:16.359 --> 00:45:19.469 with chronic kidney disease in that. 00:45:19.469 --> 00:45:23.841 So very, very important results in it All right. 00:45:24.341 --> 00:45:31.671 Uh, I, I leave you with when you're designing trials, design futility. 00:45:32.281 --> 00:45:36.391 It, it-- really, it's a really important part of the, the trial design. 00:45:36.911 --> 00:45:41.881 Uh, don't leave it to the DSMB to make hard decisions, uh, in this. 00:45:42.701 --> 00:45:44.611 Help make those decisions ahead of time. 00:45:45.381 --> 00:45:47.841 I appreciate you joining me here. 00:45:47.841 --> 00:45:49.651 I'll look at several papers. 00:45:49.991 --> 00:45:53.601 Again, we can go back to the design and think about them from the results. 00:45:54.021 --> 00:45:57.511 But I appreciate you joining me here and, and joining again. 00:45:57.511 --> 00:46:01.541 And until next time, we will be here in the interim