WEBVTT NOTE This file was generated by Descript 00:00:00.000 --> 00:00:03.130 Judith: Welcome to Berry's In the Interim podcast, where we explore the 00:00:03.130 --> 00:00:07.300 cutting edge of innovative clinical trial design for the pharmaceutical and 00:00:07.300 --> 00:00:09.579 medical industries, and so much more. 00:00:09.979 --> 00:00:10.600 Let's dive in. 00:00:12.462 --> 00:00:13.722 Scott Berry: Welcome everybody. 00:00:13.722 --> 00:00:15.192 Back to, in the Interim. 00:00:15.192 --> 00:00:16.602 I'm your host, Scott Berry. 00:00:17.172 --> 00:00:20.922 Have a, uh, really honored to have my guest today. 00:00:21.282 --> 00:00:28.392 Uh, a very well-known, uh, uh, me, uh, uh, award-winning statistician 00:00:28.392 --> 00:00:32.352 with me today, we're gonna talk about a number of things, uh, with Dr. 00:00:32.352 --> 00:00:33.792 Steven Sen. 00:00:34.182 --> 00:00:37.782 He's a, he's worked as a statistician, an academic. 00:00:38.312 --> 00:00:43.592 In various positions in Switzerland, Scotland, England, Luxembourg has a 00:00:43.592 --> 00:00:49.802 really interesting history of, uh, of work from, uh, being, which is a, a, a 00:00:49.802 --> 00:00:52.952 really cool title, the head of competence. 00:00:53.177 --> 00:00:56.237 At the Center for Methodology and Statistics. 00:00:56.657 --> 00:00:58.637 Uh, I love that title. 00:00:58.937 --> 00:01:04.217 Uh, he's been a professor of Statistics at the University of Glasgow, uh, professor 00:01:04.217 --> 00:01:08.417 of Pharmaceutical and Health Statistics at University College of London. 00:01:08.627 --> 00:01:13.367 He spent, uh, eight years at cba ge, uh, as well. 00:01:13.637 --> 00:01:16.067 Um, he's a co-author, sorry. 00:01:16.067 --> 00:01:18.857 He's the author of Crossover Trials. 00:01:19.172 --> 00:01:25.202 In clinical research, statistical issues in drug development, dicing with death. 00:01:25.652 --> 00:01:32.672 He was awarded the 2009 Bradford Hill Medal of the Royal Statistical Society 00:01:32.672 --> 00:01:38.642 in 2017, the Fisher Memorial Lecture, and so I honored to have you on. 00:01:38.642 --> 00:01:42.092 Welcome to In the Interim, Steven. 00:01:42.237 --> 00:01:42.567 Stephen Senn: thanks. 00:01:42.567 --> 00:01:43.407 Thanks for the invitation. 00:01:43.407 --> 00:01:44.152 The pleasure is mine. 00:01:45.657 --> 00:01:52.077 Scott Berry: So, uh, I, my father Don Berry has, has, uh, attributed 00:01:52.077 --> 00:01:57.507 a quote to you and I'm wondering if you want to, uh, attach this. 00:01:57.747 --> 00:02:04.077 He said that you said, I don't care whether someone is Bayesian or 00:02:04.077 --> 00:02:09.477 Frequentist, as long as they understand the regression to the mean effect. 00:02:09.762 --> 00:02:14.202 Stephen Senn: Yes, I, I tend to agree that I, I, uh, I often think that 00:02:14.202 --> 00:02:18.762 it's, it's far more important to, uh, to know how the data arrived. 00:02:18.762 --> 00:02:22.092 I, I think that the, the basic thing a statistician should always ask themselves, 00:02:22.332 --> 00:02:23.982 how do I get to see what I see? 00:02:25.482 --> 00:02:29.202 And, uh, the, the, this, this, the central fact about regression to the 00:02:29.202 --> 00:02:33.852 mean is that the data you are using as a comparator were selected to be what 00:02:33.852 --> 00:02:38.652 they are and the data you are using to make the comparison or an outcome. 00:02:38.682 --> 00:02:40.392 And they're fundamentally different things. 00:02:41.022 --> 00:02:44.622 The, the patients only got into the particular trial because 00:02:44.622 --> 00:02:46.092 they had the values you defined. 00:02:46.092 --> 00:02:48.252 They wouldn't have got in if the blood pressure hadn't 00:02:48.252 --> 00:02:49.902 been above a certain level. 00:02:50.202 --> 00:02:52.872 After that, what you did was something different. 00:02:53.262 --> 00:02:57.252 You didn't set a criterion, you actually observed what happened, and that's 00:02:57.252 --> 00:03:01.512 basically, it's the asymmetry between those two things and understanding 00:03:01.512 --> 00:03:03.852 why this causes a particular problem. 00:03:03.852 --> 00:03:06.582 That sort of thing is the thing that statisticians ought to understand. 00:03:07.087 --> 00:03:07.387 Scott Berry: Hmm. 00:03:07.747 --> 00:03:08.017 Hmm. 00:03:08.352 --> 00:03:11.922 Stephen Senn: straight away why it's nonsense to claim that because, um, 00:03:12.192 --> 00:03:13.932 actors who win an Oscar live longer. 00:03:14.517 --> 00:03:18.477 That, uh, the esteem of winning an Oscar is causing them to live longer. 00:03:18.657 --> 00:03:21.657 If you can see what the fundamental flaw is there, then you're 00:03:21.657 --> 00:03:23.097 beginning to be a statistician. 00:03:23.097 --> 00:03:23.397 It doesn't 00:03:23.502 --> 00:03:23.802 Scott Berry: Yeah. 00:03:23.967 --> 00:03:25.137 Stephen Senn: a Bayesian or a frequencies. 00:03:25.167 --> 00:03:25.347 Yeah. 00:03:25.347 --> 00:03:25.677 So, 00:03:27.087 --> 00:03:27.357 Scott Berry: Yeah. 00:03:27.477 --> 00:03:27.747 Yeah. 00:03:27.747 --> 00:03:32.157 Well, and, and, and it may even come up as we talk about what is a placebo effect, 00:03:32.247 --> 00:03:34.437 uh, which I think you laid out very much. 00:03:34.737 --> 00:03:38.487 Uh, which I think is, is very much misunderstood by 00:03:38.487 --> 00:03:40.197 many in, in clinical trials. 00:03:41.032 --> 00:03:43.132 Uh, and so I, I wanna talk. 00:03:43.132 --> 00:03:48.352 We want to talk, uh, and, and you, you have been quite, uh, active 00:03:48.352 --> 00:03:50.212 on social media, by the way. 00:03:50.212 --> 00:03:55.912 I very much enjoyed the, uh, on Twitter, your pictures of the various hikes you 00:03:55.912 --> 00:04:01.462 take, uh, beautiful hikes, which always seemed to end with a picture of a beer. 00:04:01.852 --> 00:04:04.642 Uh, but, but, but those are fantastic. 00:04:04.642 --> 00:04:06.592 I hope you're able to still do those. 00:04:06.762 --> 00:04:07.212 Stephen Senn: Yes. 00:04:07.822 --> 00:04:08.512 Scott Berry: Yep, yep. 00:04:08.542 --> 00:04:09.412 I, I've, I've. 00:04:09.432 --> 00:04:10.452 Stephen Senn: be doing a hike tomorrow. 00:04:10.452 --> 00:04:11.232 The weather looks good. 00:04:11.232 --> 00:04:13.542 So I think, uh, my wife and I will do a hike tomorrow. 00:04:14.007 --> 00:04:15.087 Scott Berry: Ah, fantastic. 00:04:15.087 --> 00:04:15.867 Fantastic. 00:04:16.257 --> 00:04:16.857 Uh. 00:04:17.262 --> 00:04:21.042 Stephen Senn: I'm usually on my own uh, my wife has a walking club. 00:04:21.042 --> 00:04:21.342 She's in. 00:04:21.972 --> 00:04:23.352 Which I'm not allowed to join. 00:04:23.352 --> 00:04:27.492 So she, she goes off on a Thursday, I go do something else on my, on the 00:04:27.522 --> 00:04:31.512 Thursday, and then other days we might be doing, going off for a hike together. 00:04:31.512 --> 00:04:31.782 So. 00:04:32.277 --> 00:04:33.327 Scott Berry: Ah, that's wonderful. 00:04:33.387 --> 00:04:34.137 That's wonderful. 00:04:34.557 --> 00:04:40.137 As, as part of, uh, Steven's activity on social media, a number of really wonderful 00:04:40.137 --> 00:04:43.887 blogs, uh, I, I, I suggest you read them. 00:04:44.077 --> 00:04:49.357 Then one of them came up, which is titled Beware of the Morlocks, uh, 00:04:49.357 --> 00:04:51.997 and, and Loving Stephen's, uh, blogs. 00:04:51.997 --> 00:04:56.197 I go read it, it turns out, may, maybe a criticism of something I was 00:04:56.197 --> 00:04:57.877 involved in, so we're gonna get to that. 00:04:58.267 --> 00:05:02.317 But first, there's lots going on in the world, uh, here. 00:05:02.347 --> 00:05:04.447 Uh, the FDA guidance on. 00:05:04.682 --> 00:05:06.962 Draft guidance on Bayesian statistics. 00:05:07.292 --> 00:05:12.212 Uh, part of your beware of the morlocks talks about bays and, and, and not. 00:05:12.512 --> 00:05:16.922 I think it might be nice to throw it over to you about the bays, not 00:05:16.922 --> 00:05:21.872 bays, your position on that, and then maybe even leading into the FDA 00:05:21.872 --> 00:05:23.642 guidance and what you think of that up. 00:05:24.417 --> 00:05:26.427 Stephen Senn: Okay, so I can start with flexible designs. 00:05:26.427 --> 00:05:28.227 If you change the allocation ratio and you. 00:05:28.677 --> 00:05:30.057 Nothing else about it. 00:05:30.417 --> 00:05:35.727 The net effect will be, there will be a, an induced correlation between time and 00:05:35.877 --> 00:05:41.247 the treatment because at a particular era, more of A was given than B and another era 00:05:41.247 --> 00:05:42.957 more of B was given than a or whatever. 00:05:43.167 --> 00:05:46.797 And so basically, unless you do something about it, time is confounded 00:05:46.887 --> 00:05:49.377 with, um, the treatment's given. 00:05:50.187 --> 00:05:52.677 And so a natural thing is to try and de confound it. 00:05:53.517 --> 00:05:57.147 So the classical frequentist way is to say, every time you change the allocation 00:05:57.147 --> 00:06:03.747 ratio, you declare a stratum and you then essentially fit the stratum as an, as a 00:06:03.747 --> 00:06:08.517 fixed effect factor, and that will force the construction of all estimates on 00:06:08.517 --> 00:06:10.887 the basis of within stratum differences. 00:06:11.187 --> 00:06:14.367 The only way you can eliminate the stratum effect is by constructing a 00:06:14.367 --> 00:06:17.187 difference, first of all, and then everything basically boils down 00:06:17.187 --> 00:06:20.817 to weighted combinations of these within, within strata differences. 00:06:21.582 --> 00:06:24.012 So that, that's, that's what happens. 00:06:24.192 --> 00:06:27.582 But of course in doing that, you're using up a large number of degrees of freedom. 00:06:28.392 --> 00:06:32.382 Um, and you may ask yourself, can time really be this complicated? 00:06:33.182 --> 00:06:33.392 Scott Berry: Hmm. 00:06:33.612 --> 00:06:36.132 Stephen Senn: so the national thing to do is to say, well, maybe I 00:06:36.132 --> 00:06:40.002 could use a rather simpler function, one with rather fewer parameters. 00:06:41.052 --> 00:06:44.112 uh, if you have fewer parameters, the penalty you will pay for loss of 00:06:44.112 --> 00:06:45.732 orthogonality will be rather less. 00:06:46.062 --> 00:06:48.852 There's always a penalty for any loss of orthogonality. 00:06:48.852 --> 00:06:53.232 In any regression model, the penalty will be less and a natural way to do 00:06:53.232 --> 00:06:56.532 this, a natural way to be flexible is Bayesian, I don't actually see the 00:06:56.532 --> 00:06:58.722 Bayesian part of it as being essential. 00:06:59.202 --> 00:07:03.582 Essentially what you're doing is you're replacing a multi-parameter 00:07:04.002 --> 00:07:07.152 adjustment with something which uses rather fewer parameters. 00:07:07.152 --> 00:07:07.572 That's one. 00:07:07.597 --> 00:07:08.562 One way of putting it. 00:07:09.297 --> 00:07:10.227 So that's the first thing. 00:07:10.227 --> 00:07:12.597 The second thing is, well, it has to be a little bit careful. 00:07:12.597 --> 00:07:16.737 There's an, there is a correlation between time and allocation, but 00:07:16.737 --> 00:07:21.807 time covers a whole host of things and this is not always appreciated. 00:07:22.497 --> 00:07:27.057 the danger is you concentrate on time as if it was something continuous. 00:07:27.477 --> 00:07:32.157 Some aspects of it are, is reasonable to believe in some smoothness of time, 00:07:32.427 --> 00:07:34.377 but some things are not so continuous. 00:07:34.407 --> 00:07:36.117 And I gave some examples in the block. 00:07:36.297 --> 00:07:36.807 So that's 00:07:36.877 --> 00:07:36.997 Scott Berry: Hmm. 00:07:36.997 --> 00:07:37.397 Mm-hmm. 00:07:37.767 --> 00:07:37.887 Stephen Senn: of. 00:07:38.337 --> 00:07:39.942 What's, what's in the blog? 00:07:40.452 --> 00:07:40.962 Scott Berry: Oh, okay. 00:07:40.962 --> 00:07:43.632 So let, let's set up the blog and, and, and do that. 00:07:43.632 --> 00:07:48.852 But before I, I wonder if before we do that, just the, the, the Bayesian in 00:07:48.852 --> 00:07:53.832 frequentist thing, I given the quote that, that, um, uh, about regression 00:07:53.832 --> 00:07:59.472 to the mean, you are somewhat agnostic to this, and this is not anti basian. 00:07:59.802 --> 00:08:01.812 This is not anti frequentist. 00:08:01.812 --> 00:08:04.797 This, this is about functionality of the model. 00:08:05.312 --> 00:08:05.937 Stephen Senn: Yeah, sure. 00:08:06.027 --> 00:08:09.447 If, if what you did was you had an uninformative prior on each of these 00:08:09.447 --> 00:08:14.247 stratum effects, you would effectively be fitting a fixed effect frequentist model. 00:08:14.772 --> 00:08:14.992 Scott Berry: Hmm. 00:08:15.267 --> 00:08:19.677 Stephen Senn: It's only because you do something like, um, you either have a. 00:08:20.172 --> 00:08:23.472 A polynomial function in which you do something like you 00:08:23.472 --> 00:08:25.422 penalize higher order terms. 00:08:25.477 --> 00:08:30.162 You, you, you make them less likely to be, to be as large as lower order 00:08:30.162 --> 00:08:35.562 terms or you, uh, you imagine that you have somehow some pseudo data that you 00:08:35.562 --> 00:08:39.552 can add in some particular way to help with the, with a particular adjustment. 00:08:39.552 --> 00:08:43.392 It's only if you start doing that, um, which is a perfectly natural 00:08:43.392 --> 00:08:45.042 thing to do for anybody modeling. 00:08:45.447 --> 00:08:48.687 That, that, that, that then arises what I think is dangerous. 00:08:48.687 --> 00:08:52.737 If people say, oh, because I'm using bays, the problem is solved. 00:08:52.737 --> 00:08:56.397 There are still very, very hard decisions to be made in principle 00:08:56.397 --> 00:08:58.587 about how were you going to use bays. 00:08:58.647 --> 00:08:59.697 That's the big issue, 00:09:00.087 --> 00:09:00.267 Scott Berry: Hmm. 00:09:00.567 --> 00:09:01.017 Stephen Senn: whether. 00:09:01.722 --> 00:09:01.842 Scott Berry: Yeah. 00:09:02.412 --> 00:09:06.432 Okay, so let, let's set up the problem and, and, and maybe take one step back. 00:09:06.432 --> 00:09:10.272 Just if, if you read Steven's blog, beware the Morelocks so you can 00:09:10.272 --> 00:09:11.922 get there on his LinkedIn page. 00:09:12.282 --> 00:09:17.952 But, but to set up the problem is, uh, and, and you discuss the, the Seville 00:09:17.952 --> 00:09:22.182 paper, which I'm a co-author on the night, uh, talks about the Bayesian time machine. 00:09:22.587 --> 00:09:29.007 And so the A scenario I think that we, we can use is that you have a platform trial. 00:09:29.427 --> 00:09:34.527 And the platform trial starts with a common control and you have one 00:09:34.527 --> 00:09:38.697 experimental arm and you make reference to one of the figures on there. 00:09:39.027 --> 00:09:44.277 You have one experimental arm that starts, so it's arm one against control for 00:09:44.277 --> 00:09:47.547 the first, there's 10 periods of time. 00:09:47.657 --> 00:09:52.907 Within this graph, uh, by the way, this is the challenge of a, uh, of a podcast 00:09:52.907 --> 00:09:56.807 is try to describe to people without actually showing them a graph, uh, 00:09:56.837 --> 00:10:02.297 where arm one and control are enrolling equally one-to-one, and then a new arm 00:10:02.297 --> 00:10:05.327 is added in time three, arm two is added. 00:10:05.327 --> 00:10:09.017 Arm one is still being used, and our and control is being used. 00:10:09.107 --> 00:10:13.127 And now it's one to one to one for control. 00:10:13.127 --> 00:10:14.237 Arm one and arm two. 00:10:14.892 --> 00:10:18.462 That continues in period four, in period five. 00:10:18.462 --> 00:10:20.232 Arm one goes away. 00:10:20.412 --> 00:10:22.392 It's done enrolling patients. 00:10:22.752 --> 00:10:23.832 Arm two is there. 00:10:23.832 --> 00:10:28.242 Now we add arm three, we add arm four, arm five eventually 00:10:28.242 --> 00:10:29.682 adds through the 10 period. 00:10:29.682 --> 00:10:34.177 So you can imagine staggered arms in the trial enroll. 00:10:34.932 --> 00:10:39.822 A common control throughout being randomized during the 00:10:39.822 --> 00:10:42.672 trial, uh, within this setting. 00:10:43.002 --> 00:10:47.562 And then the question comes down about how to make inferences about 00:10:47.562 --> 00:10:50.892 one of the arms relative to control. 00:10:52.632 --> 00:10:56.712 One of the, the, the simplest thing would probably be to just compare 00:10:56.712 --> 00:11:01.422 that arm to only the concurrent controls, and let's avoid for a second. 00:11:01.422 --> 00:11:04.572 You talk about having multiple placebos. 00:11:04.632 --> 00:11:07.392 Maybe avoid that for a second, but I think that's something you 00:11:07.392 --> 00:11:08.832 importantly want to talk about. 00:11:09.852 --> 00:11:14.982 So when we're making inferences about a particular arm, do we compare that arm to 00:11:14.982 --> 00:11:20.322 the common controls that were randomized and eligible for that control at the time? 00:11:21.357 --> 00:11:25.557 Do we use other controls that have been enrolled, say 00:11:25.557 --> 00:11:28.317 before that ARM was enrolled? 00:11:28.737 --> 00:11:34.587 So you have a control patient that was randomized in the same trial at the 00:11:34.587 --> 00:11:38.967 time you're, if we're talking about ARM two, it was randomized when ARM 00:11:38.967 --> 00:11:43.857 one was there and it was a control patient, but before ARM two was there. 00:11:44.787 --> 00:11:47.667 So that's considered a non concurrent control. 00:11:48.312 --> 00:11:48.702 Stephen Senn: Right. 00:11:49.002 --> 00:11:52.692 Scott Berry: And so are we going to use that in some way 00:11:52.692 --> 00:11:54.462 to make inference about arm two? 00:11:55.512 --> 00:11:58.152 Is that a reasonable setup of the problem? 00:11:58.512 --> 00:12:00.972 Stephen Senn: Yeah, I think that's, uh, that's reasonable. 00:12:01.182 --> 00:12:06.702 Um, and the, there is a relationship to incomplete block designs where 00:12:06.702 --> 00:12:10.182 typically in an incomplete block design, not every block gets every treatment. 00:12:10.182 --> 00:12:11.802 I mean, that's basically why it's incomplete. 00:12:12.072 --> 00:12:12.342 Scott Berry: hmm. 00:12:12.432 --> 00:12:16.932 Stephen Senn: And the sort of tradition there in, um, agricultural 00:12:16.932 --> 00:12:20.592 statistics, where they were often used was that they had to be connected. 00:12:21.492 --> 00:12:25.572 You had to somehow be able to construct the, um, the treatment 00:12:25.572 --> 00:12:28.752 effect as on the basis of a number of within block differences. 00:12:29.082 --> 00:12:33.912 However, um, later I think it was Yates realized there was 00:12:33.912 --> 00:12:37.152 some further information that was recoverable in another way if 00:12:37.152 --> 00:12:38.562 you'd randomized between blocks. 00:12:39.482 --> 00:12:39.902 Scott Berry: Mm-hmm. 00:12:40.677 --> 00:12:43.917 Stephen Senn: But bas basically they had to be connected in order to, in order 00:12:43.917 --> 00:12:47.667 to actually make the comparison using all the information that there was. 00:12:48.282 --> 00:12:48.342 Scott Berry: Oh, 00:12:48.537 --> 00:12:51.747 Stephen Senn: you were just limited to those particular blocks in which the 00:12:51.747 --> 00:12:54.417 two, the pair of treatments you were interested in happened to be represented. 00:12:54.792 --> 00:12:55.002 Scott Berry: right. 00:12:55.272 --> 00:13:00.012 Uh, and, and in a lot of these platform trials, you get this multiple overlapping 00:13:00.012 --> 00:13:03.117 or bridging where when ARM two was there. 00:13:03.807 --> 00:13:08.367 Uh, it was there with arm one and control and so earlier, arm one and control 00:13:08.367 --> 00:13:11.127 provide some potential information. 00:13:11.127 --> 00:13:11.397 So 00:13:11.757 --> 00:13:16.737 you, we, we could do only comparing to the concurrent controls, 00:13:17.457 --> 00:13:21.717 we could do, as you described, where we put in a, uh, covariate 00:13:21.717 --> 00:13:22.527 for each piece of time. 00:13:23.487 --> 00:13:26.727 Uh, in a, in a frequentist fixed effect. 00:13:26.727 --> 00:13:30.537 And we add in, we add in nine degrees of freedom or something 00:13:30.537 --> 00:13:32.187 like that for, for, for time. 00:13:32.517 --> 00:13:37.677 Um, within that, what the Bayesian time machine or the paper, the Seville paper 00:13:37.677 --> 00:13:42.807 talks about, which will come back to the guidance, but the FDA guidance references 00:13:42.807 --> 00:13:45.087 this paper and, and a couple trials. 00:13:45.087 --> 00:13:47.817 It's used in GBM Agile, for example. 00:13:48.087 --> 00:13:48.537 Um. 00:13:49.167 --> 00:13:53.997 That it takes those units of time because they are ordered 00:13:54.867 --> 00:13:56.247 chronologically. 00:13:56.427 --> 00:14:00.927 It does a smoothing estimate of the effect of time over these blocks. 00:14:01.257 --> 00:14:04.767 Um, coming back a little bit to the regression effect, but 00:14:04.767 --> 00:14:06.567 that this smoothing over time. 00:14:06.567 --> 00:14:06.867 Yes. 00:14:06.867 --> 00:14:13.107 It uses largely a, a, a smoothing spline over time, and that's the reference 00:14:13.107 --> 00:14:16.257 to the time machine and it's done in a Bayesian way, but as you say, this 00:14:16.257 --> 00:14:17.497 could be done in a frequentist way. 00:14:18.232 --> 00:14:18.522 Stephen Senn: Yeah. 00:14:18.527 --> 00:14:18.627 Yeah. 00:14:19.377 --> 00:14:19.677 Uh, the 00:14:19.677 --> 00:14:20.282 Scott Berry: So y 00:14:21.327 --> 00:14:23.967 Stephen Senn: There would be a slight, a problem in in frequentist one, which 00:14:24.147 --> 00:14:32.787 some Bayesian methods could avoid, is that the polynomial you could fit change 00:14:32.787 --> 00:14:34.077 as the number of periods were added. 00:14:35.502 --> 00:14:39.162 Obviously very early on you can't, you can't fit a polynomial 00:14:39.162 --> 00:14:41.112 with four or five parameters. 00:14:41.112 --> 00:14:41.382 Four or 00:14:41.397 --> 00:14:41.667 Scott Berry: hmm. 00:14:41.682 --> 00:14:43.842 Stephen Senn: is still less than the nine or whatever that you would 00:14:43.842 --> 00:14:45.912 use for a full frequent as fit. 00:14:45.912 --> 00:14:47.622 But earlier on you couldn't do that. 00:14:47.862 --> 00:14:50.772 So potentially you'd actually be smoothing because as you went 00:14:50.772 --> 00:14:54.022 later on, you could actually revise some of your smoothing things. 00:14:54.022 --> 00:14:58.092 You'd actually be smoothing in a slightly different way as you went further on. 00:14:58.782 --> 00:14:59.002 Scott Berry: Hmm. 00:14:59.862 --> 00:15:04.482 Stephen Senn: now, it's inherent to, to Bayesian approaches that you 00:15:04.482 --> 00:15:07.872 learn as you go along anyway, so, you know, that's not necessarily 00:15:07.872 --> 00:15:09.132 seen as being a big deal there. 00:15:10.827 --> 00:15:15.267 Scott Berry: Okay, so this, uh, in, in some of these platform trials, when you 00:15:15.267 --> 00:15:21.447 use this model and we're making inferences about arm two, it does use controls 00:15:21.447 --> 00:15:28.242 from earlier in their comparison to arm one, which is also there with arm two. 00:15:28.317 --> 00:15:32.937 So it, it, it does use these essentially adjusting for time. 00:15:33.897 --> 00:15:39.057 And we have a quote in that paper, which you put in your blog where we 00:15:39.057 --> 00:15:41.547 say, uh, and there's two parts to this. 00:15:41.547 --> 00:15:43.347 I wanna talk about that largely. 00:15:43.887 --> 00:15:46.887 Uh, we're, we're in a world right now where you hear a ton about 00:15:46.887 --> 00:15:49.407 real world data, historical data. 00:15:49.647 --> 00:15:54.207 You could go out and get historical controls for a glioblastoma 00:15:54.207 --> 00:15:58.617 trial and use those to help make inferences in your trial. 00:15:58.617 --> 00:15:59.187 You're running now. 00:16:00.107 --> 00:16:03.557 That data's from a different protocol, different data. 00:16:03.707 --> 00:16:08.327 It's got a lot of things that a non concurrent control in a 00:16:08.327 --> 00:16:10.457 platform trial doesn't suffer from. 00:16:11.327 --> 00:16:12.377 Uh, we do. 00:16:12.377 --> 00:16:15.737 So it's the same protocol, the same data elements. 00:16:15.737 --> 00:16:18.347 All of these controls were randomized. 00:16:18.707 --> 00:16:21.552 Uh, now it was at a different time, so we say. 00:16:21.987 --> 00:16:24.087 Time is the thing that's different. 00:16:24.537 --> 00:16:28.947 You brought up that there's more than time and I, I want you to maybe touch 00:16:28.947 --> 00:16:31.347 on this again, that it's not just time. 00:16:32.337 --> 00:16:32.577 Stephen Senn: No. 00:16:32.577 --> 00:16:33.987 So let me, let me give you an example. 00:16:33.987 --> 00:16:39.927 If you were to have a look at, um, of NEVIRAPINE in, uh, HIV infection, a 00:16:39.927 --> 00:16:41.457 lot of them were placebo controlled. 00:16:42.387 --> 00:16:43.227 But what did that mean? 00:16:43.227 --> 00:16:47.157 It actually meant that because a ZT was already approved as a treatment 00:16:47.157 --> 00:16:50.302 for HIV infection, it meant that the patients were all getting a ZT. 00:16:51.537 --> 00:16:54.357 But some of them, in addition, got placebo to Nevirapine and some 00:16:54.357 --> 00:16:56.127 of them got ine in in addition. 00:16:56.607 --> 00:16:59.637 And some people would describe that as being a placebo controlled trial. 00:16:59.637 --> 00:17:01.497 I would, others would say, no, no, no. 00:17:01.497 --> 00:17:05.997 It's a trial of the combination therapy of NEVIRAPINE and a ZT versus 00:17:05.997 --> 00:17:11.517 a ZT alone doesn't matter all the time in clinical trials all the 00:17:11.517 --> 00:17:12.987 time there is background therapy. 00:17:13.617 --> 00:17:15.897 All the time, standard of care is evolving. 00:17:16.377 --> 00:17:19.497 So when you said just now that the protocol is the same, yes. 00:17:19.557 --> 00:17:25.017 But the world doesn't stand still actually in any particular serious 00:17:25.017 --> 00:17:29.487 disease, what you will find is all the time that your trial is running, then 00:17:29.487 --> 00:17:31.647 in that case, the world is evolving. 00:17:31.647 --> 00:17:35.997 And to get return to the HIV trials, we know that people who were 00:17:35.997 --> 00:17:38.787 recruited later into the same trial. 00:17:39.357 --> 00:17:42.327 And given the same treatment, had better survival than those 00:17:42.327 --> 00:17:43.437 who were recruited earlier. 00:17:43.737 --> 00:17:44.517 Why was that? 00:17:44.757 --> 00:17:48.567 It was because you were learning more about the treatment of AIDS as time 00:17:48.567 --> 00:17:53.757 went on and all the patients in those trials continue to benefit from the 00:17:53.757 --> 00:17:55.917 improvement in care that was going on. 00:17:57.327 --> 00:17:58.587 yes, I agree with you. 00:17:58.592 --> 00:18:02.517 It's time in a sense, but I don't agree that the fact that the same 00:18:02.517 --> 00:18:06.297 protocols are used deals with all the problems, not by a long way. 00:18:07.062 --> 00:18:09.822 Scott Berry: Okay, so part of it is all the problems. 00:18:10.332 --> 00:18:14.922 Uh, but the other part is some of the problems, uh, as, as a comparison 00:18:14.922 --> 00:18:17.712 of external data to some extent. 00:18:17.892 --> 00:18:21.192 It's a much higher level of that, but doesn't, yeah. 00:18:21.252 --> 00:18:21.552 Okay. 00:18:22.002 --> 00:18:22.182 Okay. 00:18:22.197 --> 00:18:23.907 Stephen Senn: but it doesn't, it doesn't deal with all that. 00:18:23.907 --> 00:18:24.267 And then I 00:18:24.432 --> 00:18:24.792 Scott Berry: Right. 00:18:25.647 --> 00:18:27.567 Stephen Senn: one in which, an earlier paper of yours, um. 00:18:28.572 --> 00:18:33.672 rightly discussed, but this later Savile paper didn't, um, that you discussed 00:18:33.672 --> 00:18:37.302 the fact that it will be impossible to blind all of the treatments to each 00:18:37.302 --> 00:18:40.452 other, uh, in these particular trials, 00:18:40.632 --> 00:18:40.872 Scott Berry: so 00:18:41.382 --> 00:18:42.972 let I, let's not move on to that yet. 00:18:43.002 --> 00:18:43.302 But the, 00:18:43.607 --> 00:18:44.217 okay, so, 00:18:44.232 --> 00:18:45.132 Stephen Senn: get onto that later then. 00:18:45.132 --> 00:18:45.312 Yeah. 00:18:45.447 --> 00:18:45.737 Scott Berry: okay, 00:18:46.182 --> 00:18:50.412 so in, in the setting here, let's think of this as a common control. 00:18:50.592 --> 00:18:53.952 Uh, and not that there's different modes of administration 00:18:53.952 --> 00:18:55.212 of a, of a, of a control. 00:18:55.422 --> 00:18:56.892 Stephen Senn: you one, one other question. 00:18:57.252 --> 00:18:57.472 Scott Berry: Um, 00:18:57.477 --> 00:19:00.477 Stephen Senn: It's not the case, I don't think, but correct me if I'm 00:19:00.477 --> 00:19:04.257 wrong, that you will necessarily stick with all the centers through 00:19:04.257 --> 00:19:05.847 all the life of the platform trial 00:19:06.657 --> 00:19:07.947 Scott Berry: yeah, one, but. 00:19:08.037 --> 00:19:10.077 Stephen Senn: not the case in a standard parallel group trial. 00:19:10.107 --> 00:19:13.947 It's, I've been sat on many data monitoring boards and you find that 00:19:13.947 --> 00:19:17.967 recruitment is poor and the sponsor says, oh, well, we'll enroll some new centers. 00:19:18.747 --> 00:19:22.557 the enrolling of new centers, the dropping of new centers occurs all the time. 00:19:22.557 --> 00:19:24.837 So your differences in time. 00:19:25.197 --> 00:19:27.627 Are also differences between centers 00:19:28.212 --> 00:19:28.302 Scott Berry: Hmm. 00:19:29.022 --> 00:19:29.382 Yep. 00:19:29.457 --> 00:19:31.587 Stephen Senn: a parallel group trial as you would treat a 00:19:31.587 --> 00:19:32.907 cluster randomized trial. 00:19:33.207 --> 00:19:36.657 So therefore, this center effect is something you have to deal with. 00:19:36.687 --> 00:19:37.257 It's dealt 00:19:37.392 --> 00:19:37.602 Scott Berry: Yeah. 00:19:38.697 --> 00:19:41.757 Stephen Senn: in a standard, fixed, uh, allocation trial. 00:19:41.847 --> 00:19:44.517 It's not dealt with in a platform trial. 00:19:45.177 --> 00:19:45.357 Scott Berry: Yeah. 00:19:45.357 --> 00:19:50.937 One of the nice things about many standing trials, the IY two, uh, 00:19:50.967 --> 00:19:55.317 neoadjuvant breast cancer, that trial ran for 10 years essentially. 00:19:55.947 --> 00:20:01.257 It started with 20 sites and these sites, sites lasted for 10 years 00:20:01.257 --> 00:20:03.357 with very little changes to them. 00:20:03.357 --> 00:20:03.717 So 00:20:03.987 --> 00:20:08.367 one of the benefits, right, one of the benefits of the platform becomes almost 00:20:08.367 --> 00:20:10.227 this learning healthcare thing to that. 00:20:10.377 --> 00:20:13.617 Now you, you're right that there is some variation to that and 00:20:13.617 --> 00:20:17.817 a number of trials we do try to adjust for, for, for center site. 00:20:18.507 --> 00:20:20.902 One of the things about this though, as you mentioned. 00:20:21.192 --> 00:20:24.552 Time and, and maybe this is semantics about whether time 00:20:24.552 --> 00:20:29.142 includes the evolution of background care and other aspects to it. 00:20:29.622 --> 00:20:34.242 But we also empirically have these observations on the control 00:20:34.242 --> 00:20:36.672 arm over these 10 periods. 00:20:37.122 --> 00:20:42.612 So we empirically see that, uh, outcomes are getting better over time. 00:20:43.032 --> 00:20:49.152 Now there's absolutely the assumption of additivity of that across arms. 00:20:50.127 --> 00:20:55.227 So within the model that an additive effect, whether this is a hazard 00:20:55.227 --> 00:21:00.987 ratio, whether this is a a, a responder analysis empirically, we can see 00:21:00.987 --> 00:21:04.557 whether or not this is happening during the course of the trial. 00:21:05.242 --> 00:21:05.532 Yeah. 00:21:05.742 --> 00:21:08.217 Stephen Senn: I, I, uh, I have no problem with that. 00:21:08.217 --> 00:21:09.027 And in any case, I 00:21:09.117 --> 00:21:09.177 Scott Berry: Yeah. 00:21:09.177 --> 00:21:09.187 Yeah. 00:21:09.237 --> 00:21:11.187 Stephen Senn: really regard the activity assumption as 00:21:11.187 --> 00:21:12.867 being particularly important. 00:21:13.887 --> 00:21:15.297 I'm less concerned about that. 00:21:15.597 --> 00:21:19.197 I regard that as being a treatment by time interaction. 00:21:20.282 --> 00:21:23.367 what I'm really interested in is the main effective time, and I think the 00:21:23.522 --> 00:21:23.762 Scott Berry: Mm-hmm. 00:21:23.942 --> 00:21:26.997 Stephen Senn: of time is something that can be underestimated 00:21:27.057 --> 00:21:28.137 if we're not careful. 00:21:28.622 --> 00:21:29.042 Scott Berry: Mm-hmm. 00:21:29.382 --> 00:21:32.922 Stephen Senn: Um, but essentially you're, you're replacing a, you're 00:21:32.922 --> 00:21:36.432 replacing a model with many parameters, with one with fewer parameters. 00:21:36.432 --> 00:21:40.002 That's not necessarily an unreasonable thing to do, but I'm not sure that 00:21:40.002 --> 00:21:44.442 everybody who's involved in A-D-S-M-B understands exactly what's going on. 00:21:44.832 --> 00:21:50.712 And in particular, um, the question is how should data that they use for monitoring, 00:21:51.042 --> 00:21:52.722 monitoring be presented to them? 00:21:53.377 --> 00:21:53.617 Scott Berry: Yeah. 00:21:54.037 --> 00:21:59.827 So, uh, a huge, a huge issue then is during the course of the trial, the DSMB 00:21:59.827 --> 00:22:03.577 is reviewing that there, there's this model that's making time adjustments 00:22:03.847 --> 00:22:07.387 and do they just accept that, Hey, the model's got this, don't worry about it. 00:22:07.387 --> 00:22:08.797 Or they, are they able to view this? 00:22:08.797 --> 00:22:09.522 Are they able to see it? 00:22:10.182 --> 00:22:12.432 Um, uh, uh, hugely important. 00:22:12.432 --> 00:22:15.642 Let me come back to what you said about a other type of trial, 00:22:15.642 --> 00:22:19.482 and I'll make reference to the Heal a LS trial, which has the 00:22:19.482 --> 00:22:21.222 components that you just brought up. 00:22:21.672 --> 00:22:27.012 In many trials, what happens is there isn't a common control oncology, 00:22:27.012 --> 00:22:30.162 for example, there's a standard of care and patients generally aren't 00:22:30.162 --> 00:22:32.652 even blinded in oncology trials. 00:22:33.187 --> 00:22:35.737 Because of the intensity of the treatments. 00:22:36.037 --> 00:22:44.827 But in an A, in the a LS platform trial, a patient is randomized to say drug A, B, C, 00:22:45.127 --> 00:22:50.767 and then they're further randomized three to one, to its placebo or its active. 00:22:51.807 --> 00:22:56.847 So at any time, if there's a, B, c, enrolling in the trial, patients 00:22:56.847 --> 00:23:02.307 are being randomized to a's placebo, B's, placebos, or C'S placebos. 00:23:02.907 --> 00:23:06.147 They are not blinded to A, B, or C. 00:23:06.297 --> 00:23:12.027 So if one of them is three a day or a pill and one of them is a. 00:23:12.972 --> 00:23:15.942 Uh, uh, subcutaneous shot. 00:23:16.092 --> 00:23:18.672 They're not given the blinding subcutaneous. 00:23:18.672 --> 00:23:21.192 They're only given the mode of administration of 00:23:21.192 --> 00:23:22.152 the drug that they're on. 00:23:22.812 --> 00:23:27.162 So at any one time, we have modes of administration of placebo. 00:23:27.162 --> 00:23:31.302 So not only do we have controls in that had a LS. 00:23:31.862 --> 00:23:37.772 That were enrolled slightly before the arm came on, but we also have 00:23:37.772 --> 00:23:43.172 placebos at the same time that are given different modes of administration of 00:23:43.172 --> 00:23:45.842 a placebo across the different arms. 00:23:46.052 --> 00:23:47.877 And I think you wanted to, to talk about the. 00:23:48.867 --> 00:23:49.317 Stephen Senn: Yes. 00:23:49.377 --> 00:23:52.677 I mean, I think that I've referred to such trials a long time ago. 00:23:52.677 --> 00:23:58.887 I'm trying to remember when it was, um, as veiled, uh, if you, if you don't know what 00:23:58.887 --> 00:24:02.637 treatment you are getting, but you know, some of the treatments you're not getting, 00:24:03.627 --> 00:24:07.677 in that case, um, it's not fully blind. 00:24:07.767 --> 00:24:08.247 It's sort 00:24:08.307 --> 00:24:08.727 Scott Berry: Mm-hmm. 00:24:08.757 --> 00:24:09.897 Stephen Senn: in that particular way. 00:24:10.197 --> 00:24:11.727 Oh, I use the term veiled as being, 00:24:11.747 --> 00:24:12.167 Scott Berry: Mm-hmm. 00:24:12.357 --> 00:24:13.017 Stephen Senn: obstructed. 00:24:13.677 --> 00:24:14.067 Um. 00:24:14.367 --> 00:24:16.287 I just said to have a look. 00:24:17.757 --> 00:24:18.387 Uh, 00:24:18.552 --> 00:24:19.062 Scott Berry: term. 00:24:19.122 --> 00:24:19.452 Yep. 00:24:19.497 --> 00:24:22.227 Stephen Senn: yeah, I think it might have been, 00:24:24.687 --> 00:24:26.337 yeah, 2004 I think it 00:24:26.657 --> 00:24:27.402 Scott Berry: Ah, okay. 00:24:27.687 --> 00:24:29.817 Stephen Senn: and I was thinking of particular trial that we had 00:24:29.817 --> 00:24:33.057 run at Cbga, where we had two patches of hormone replacement 00:24:33.057 --> 00:24:34.767 therapy, a high dose and a low dose. 00:24:35.187 --> 00:24:37.137 And that meant the patches were of different sizes. 00:24:37.932 --> 00:24:38.202 Scott Berry: Hmm. 00:24:38.337 --> 00:24:41.367 Stephen Senn: the only ca way you could have blinded the patients would've 00:24:41.367 --> 00:24:44.607 been by giving them two patches, a large patch and a small patch. 00:24:45.357 --> 00:24:48.807 one of them would've been, let's say, active and one of them would've been a 00:24:48.807 --> 00:24:50.487 placebo, but they wouldn't know which. 00:24:50.757 --> 00:24:52.797 And then you could maybe have had a placebo group with two 00:24:52.797 --> 00:24:54.897 patches, which were both placebos. 00:24:55.377 --> 00:25:01.527 Um, then basically, uh, a patient who's being given the highest dose knows that 00:25:01.527 --> 00:25:03.057 they're not being given the lowest dose. 00:25:04.137 --> 00:25:08.127 So if expectation leads them to report side effects, 'cause they 00:25:08.127 --> 00:25:11.007 say, wow, I'm getting a high dose of hormones, this could be a problem. 00:25:11.277 --> 00:25:15.267 I don't feel so well, and they report it, you only control for that by 00:25:15.267 --> 00:25:17.427 comparing them to their own placebo. 00:25:18.057 --> 00:25:21.807 If you compare them to the whole, the pool placebo group, then in that case 00:25:21.807 --> 00:25:23.697 you don't actually control for this. 00:25:24.177 --> 00:25:24.477 Scott Berry: Hmm. 00:25:24.927 --> 00:25:25.167 Hmm. 00:25:25.317 --> 00:25:29.007 Stephen Senn: study I was involved in was a target study where, um. 00:25:29.727 --> 00:25:32.367 OC was, uh, treatment for rheumatism. 00:25:32.367 --> 00:25:37.257 Osteoarthritis was compared to, uh, Naproxen twice daily 00:25:37.347 --> 00:25:39.297 or ibuprofen three times daily. 00:25:39.987 --> 00:25:41.607 again, it was awkward to blind things. 00:25:41.607 --> 00:25:46.197 So basically you had a substudy, which was Lum, coxib versus Naproxen, and 00:25:46.197 --> 00:25:52.437 another substudy, which was Ibuprofen versus um, uh, ibuprofen versus li 00:25:53.427 --> 00:25:56.037 And on the day safety monitoring Board, we had to take great care. 00:25:56.907 --> 00:26:03.867 To make sure we only looked at placebo, sorry, the control patients from the same 00:26:03.867 --> 00:26:07.737 sub study because the results in the two sub studies were just not comparable. 00:26:08.817 --> 00:26:09.087 So 00:26:09.157 --> 00:26:10.152 Scott Berry: so Oh, interesting. 00:26:10.232 --> 00:26:13.797 Stephen Senn: you, so you would've got, uh, a bias, you would've been biased 00:26:13.827 --> 00:26:18.297 in actually doing the monitoring if you hadn't split them in sub studies. 00:26:18.297 --> 00:26:21.987 We had to do, uh, essentially treat them as two separate trials. 00:26:23.742 --> 00:26:27.957 Scott Berry: So do you, in that case, the outcome sounds like it was 00:26:28.177 --> 00:26:30.397 osteoarthritis, pain, for example. 00:26:31.062 --> 00:26:33.762 Stephen Senn: Well, yes, although, to be honest, the, the, the trial 00:26:33.762 --> 00:26:39.252 was also looking at, um, because, um, the second generation of, uh, 00:26:39.282 --> 00:26:44.022 COX inhibitors instead of Cox two inhibitors, were supposed to be better 00:26:44.022 --> 00:26:46.182 in terms of, uh, gastric side effects. 00:26:46.872 --> 00:26:46.952 Scott Berry: Hmm. 00:26:47.322 --> 00:26:49.152 Stephen Senn: one of the things one was looking at was gastric 00:26:49.152 --> 00:26:52.992 side effects, but also there was a question mark over cardiotoxicity. 00:26:53.352 --> 00:26:57.312 So one of the other things one was looking at was, uh, cardiovascular side effects. 00:26:57.387 --> 00:26:57.597 Scott Berry: Hmm. 00:26:57.852 --> 00:27:01.152 Stephen Senn: Um, but the problem was that the two substudies were not comparable. 00:27:01.887 --> 00:27:02.157 Scott Berry: Hmm. 00:27:02.532 --> 00:27:03.582 Stephen Senn: same protocol. 00:27:03.642 --> 00:27:06.702 The only thing that was different in the protocol was essentially 00:27:06.702 --> 00:27:07.467 the treatments that were given. 00:27:08.667 --> 00:27:08.997 Scott Berry: Okay. 00:27:09.717 --> 00:27:14.277 Uh, so do, do you think that that's context specific? 00:27:14.277 --> 00:27:18.507 So for example, in the a LS trial, we have at the same time 00:27:18.507 --> 00:27:20.217 patients that are given different. 00:27:20.787 --> 00:27:26.997 Uh, placebo by different randomization endpoints are functional rating scales. 00:27:26.997 --> 00:27:30.657 They are mortality, uh, combined together. 00:27:31.077 --> 00:27:35.427 Uh, we, by the way, we have randomized comparisons of those 00:27:35.427 --> 00:27:37.227 different controls in the trial. 00:27:37.617 --> 00:27:44.097 So if you're making inference about drug A and it has its placebos, I would 00:27:44.097 --> 00:27:48.987 shudder to ignore placebo B and placebo c. 00:27:49.812 --> 00:27:52.962 First of all, we have randomized comparison and actively are 00:27:52.962 --> 00:27:54.552 they responding differently. 00:27:54.912 --> 00:27:58.812 But in some diseases, and we talked about early on, the regression of mean 00:27:58.812 --> 00:28:04.272 and what is a placebo effect, and much of the placebo effect is protocol driven 00:28:04.272 --> 00:28:08.622 and not thought that, I'm thought that I'm taking two pills a day instead of 00:28:08.622 --> 00:28:12.522 three pills a day are gonna affect the time of mortality in an a LS trial. 00:28:13.162 --> 00:28:14.682 Stephen Senn: Yes, that's, that's, that's true. 00:28:14.892 --> 00:28:20.412 And, uh, I don't necessarily argue against using a, um, let's say 00:28:20.412 --> 00:28:22.062 a model with fewer parameters. 00:28:22.242 --> 00:28:22.362 I'm 00:28:22.542 --> 00:28:22.812 Scott Berry: Hmm 00:28:22.842 --> 00:28:23.592 Stephen Senn: that that's the case. 00:28:23.592 --> 00:28:26.472 I'm just saying that these particular issues are not necessarily discussed. 00:28:26.622 --> 00:28:27.342 For example. 00:28:27.387 --> 00:28:27.807 Scott Berry: mm-hmm. 00:28:28.167 --> 00:28:29.787 Stephen Senn: To return to the time machine. 00:28:29.847 --> 00:28:33.447 Uh, in your particular SAVI Al paper, you're looking at comparing a 00:28:33.447 --> 00:28:38.007 model, which essentially has got nine parameters for time with your spline 00:28:38.007 --> 00:28:39.597 model, which has got rather fewer. 00:28:39.737 --> 00:28:40.087 Scott Berry: Right, 00:28:40.662 --> 00:28:40.992 right. 00:28:41.637 --> 00:28:44.397 Stephen Senn: uh, if everything's okay, then you're gonna do better 00:28:44.397 --> 00:28:46.917 with the model, which has got fewer parameters, no question. 00:28:47.847 --> 00:28:51.657 Um, I mean, if, if you could ignore time altogether, you'd be even better in 00:28:51.702 --> 00:28:52.152 Scott Berry: Yes. 00:28:52.347 --> 00:28:53.097 Stephen Senn: model, you know, but 00:28:53.292 --> 00:28:53.682 Scott Berry: Right, right. 00:28:53.727 --> 00:28:54.807 Stephen Senn: nobody's gonna go that far. 00:28:55.287 --> 00:28:56.607 But actually. 00:28:57.567 --> 00:28:59.637 I could argue there are not 10 groups. 00:28:59.697 --> 00:29:01.137 There are 24 groups. 00:29:01.827 --> 00:29:07.497 you look at the combination of time, period, and control taking into amount, 00:29:07.647 --> 00:29:11.217 uh, into account the blinding thing, you don't end up with 10 groups. 00:29:11.217 --> 00:29:12.687 You end up with 24. 00:29:13.047 --> 00:29:17.487 And in that case, the degree of adjustment is going to penalize you a lot more. 00:29:18.212 --> 00:29:18.432 Scott Berry: Hmm. 00:29:18.507 --> 00:29:23.307 Stephen Senn: So, so it's not even true that the, uh, I forget what 00:29:23.307 --> 00:29:25.737 you call it, the, the, the time 00:29:26.037 --> 00:29:27.537 Scott Berry: Time categorical where? 00:29:27.622 --> 00:29:27.912 Yeah. 00:29:27.912 --> 00:29:28.152 Right. 00:29:28.162 --> 00:29:28.912 Fixed effect. 00:29:29.007 --> 00:29:29.457 Stephen Senn: time, time. 00:29:29.457 --> 00:29:30.537 Categorical fixed effect. 00:29:30.807 --> 00:29:34.677 It's not even clear that the time categorical effect gets rid 00:29:34.677 --> 00:29:36.207 of all the, of all the biases. 00:29:36.207 --> 00:29:36.777 Actually, 00:29:36.872 --> 00:29:36.952 Scott Berry: Hmm. 00:29:37.017 --> 00:29:39.237 Stephen Senn: if you really believe in concurrent control, you 00:29:39.237 --> 00:29:40.857 have to have 24 groups, not 10. 00:29:41.632 --> 00:29:41.872 Scott Berry: Right. 00:29:42.267 --> 00:29:43.677 Stephen Senn: you can still make the connection. 00:29:43.857 --> 00:29:46.857 There's still a sort of connection you can probably make, but nevertheless, 00:29:46.857 --> 00:29:48.807 it's going to be a lot more difficult. 00:29:49.497 --> 00:29:50.397 Scott Berry: Right, right. 00:29:50.577 --> 00:29:54.867 Uh, and, and get that, the fully parameterized scenario in that, and 00:29:55.077 --> 00:29:59.457 is there a different effect of the placebos and does that vary over time? 00:29:59.877 --> 00:30:03.207 Starts to get hard to imagine in all of that. 00:30:03.207 --> 00:30:05.967 But that has ramifications in trial design. 00:30:06.587 --> 00:30:10.067 So when we're designing one of these trials and we do the amount 00:30:10.067 --> 00:30:14.837 of randomization to a control in that, uh, you know, forcing the 00:30:14.837 --> 00:30:19.217 fully parameterized model means we have less investigational arms. 00:30:19.217 --> 00:30:24.167 We have to enroll more patients to a placebo who have a LS for 12 00:30:24.167 --> 00:30:28.007 months, for example, has massive implications to the design. 00:30:28.337 --> 00:30:31.727 So hence the statistician plays this interesting role 00:30:32.087 --> 00:30:34.397 where it's the concern about. 00:30:34.632 --> 00:30:36.612 24 parameters, as you say. 00:30:36.732 --> 00:30:38.412 How much modeling do we do? 00:30:38.412 --> 00:30:42.162 How much are we willing to do, and the ramifications it has on the disease 00:30:42.162 --> 00:30:43.812 and the global state of treatment. 00:30:44.577 --> 00:30:44.847 Stephen Senn: Yeah. 00:30:46.212 --> 00:30:50.022 Yeah, so I, I, as I say, I'm not, I'm not arguing in always, you 00:30:50.172 --> 00:30:50.532 Scott Berry: Yep. 00:30:50.922 --> 00:30:52.782 Stephen Senn: statistics is a bias variance trade off. 00:30:52.782 --> 00:30:53.232 It's one of the 00:30:53.257 --> 00:30:53.477 Scott Berry: Yep. 00:30:53.532 --> 00:30:54.972 Stephen Senn: things, first things that you learn 00:30:55.147 --> 00:30:55.437 Scott Berry: Yeah. 00:30:55.662 --> 00:30:59.472 Stephen Senn: and you can't always come down and insist, well, I want the, uh, 00:30:59.952 --> 00:31:04.272 the unbiased solution because it really depends on how complex a model you 00:31:04.542 --> 00:31:04.782 Scott Berry: Mm 00:31:04.842 --> 00:31:06.162 Stephen Senn: what that would mean. 00:31:06.492 --> 00:31:06.792 Scott Berry: hmm. 00:31:07.062 --> 00:31:10.482 Stephen Senn: a certain degree of, um, of bias is accepted by everybody. 00:31:10.482 --> 00:31:11.172 I'm not, uh, 00:31:11.177 --> 00:31:11.297 Scott Berry: Hmm, 00:31:11.352 --> 00:31:12.342 Stephen Senn: not arguing against that. 00:31:13.152 --> 00:31:16.782 But I think there are, nevertheless, there are some things which are 00:31:16.782 --> 00:31:18.432 happening with adaptive designs. 00:31:19.362 --> 00:31:22.842 of all, I think that the claim for efficiency that was made 00:31:22.842 --> 00:31:24.012 has been somewhat misleading. 00:31:24.222 --> 00:31:26.352 I think there is a big benefit in efficiency. 00:31:26.592 --> 00:31:28.362 I think it's mainly organizational. 00:31:28.812 --> 00:31:29.592 I think it's not 00:31:29.622 --> 00:31:29.712 Scott Berry: hmm. 00:31:29.772 --> 00:31:33.792 Stephen Senn: much being able to use, um, the same controls over and over 00:31:33.792 --> 00:31:37.782 again because as you've already argued, there's actually less information 00:31:37.782 --> 00:31:39.192 in that than one might think. 00:31:41.067 --> 00:31:43.797 Of the possibility of adjusting for time effects. 00:31:44.067 --> 00:31:47.697 As soon as you start doing that, then you find the standard errors will go up. 00:31:48.627 --> 00:31:49.077 Um, 00:31:49.092 --> 00:31:53.622 Scott Berry: but, but it's sort of compared to what, compared to only 00:31:53.622 --> 00:31:58.422 looking at the concurrent controls, there can be huge advantages, um, 00:31:58.567 --> 00:32:00.882 of, of, of building that model. 00:32:00.987 --> 00:32:03.897 Stephen Senn: not, they're not as great as sometimes claimed. 00:32:04.272 --> 00:32:04.692 Scott Berry: Okay. 00:32:04.902 --> 00:32:05.412 Okay. 00:32:05.592 --> 00:32:05.952 Alright. 00:32:06.027 --> 00:32:09.357 Stephen Senn: see, you can see that from some of the, uh, the Bayesian 00:32:09.357 --> 00:32:12.567 work on using historical, uh, controls, which I like very much. 00:32:12.567 --> 00:32:14.607 I'm thinking of the sort of work that Heinz Schley 00:32:15.327 --> 00:32:15.747 Scott Berry: Mm-hmm. 00:32:16.347 --> 00:32:18.687 Stephen Senn: der and people like that based on Novartis have 00:32:18.687 --> 00:32:20.277 used for using historical data. 00:32:20.397 --> 00:32:22.017 And we, we've done a similar thing 00:32:22.067 --> 00:32:22.147 Scott Berry: Yeah. 00:32:22.197 --> 00:32:24.807 Stephen Senn: frequency mode, and what you find is you, you can 00:32:24.807 --> 00:32:29.667 identify, in one of our cases we identified 1,200 historical patients. 00:32:30.447 --> 00:32:32.637 when you looked at the between study variation. 00:32:33.162 --> 00:32:37.032 It was equivalent to having optimistically 50. 00:32:38.262 --> 00:32:40.752 So 1,200, you thought, wow, I'm rich. 00:32:40.752 --> 00:32:45.252 But actually, when you had a look at, uh, between study variation 00:32:45.252 --> 00:32:48.972 because you're using historical data, then in that case the, the 00:32:48.972 --> 00:32:50.412 information was not nearly as great. 00:32:50.712 --> 00:32:50.832 Now, 00:32:50.847 --> 00:32:51.027 Scott Berry: Hmm. 00:32:51.102 --> 00:32:54.342 Stephen Senn: saying something as drastic as that happens with platform trials, 00:32:55.122 --> 00:32:58.932 some of the, some of the discourse. 00:32:59.397 --> 00:33:04.377 Has gone in the simplistic way of saying, oh, and we can use all this control data. 00:33:04.647 --> 00:33:06.207 Well, it's not quite as simple as that, 00:33:06.702 --> 00:33:06.942 Scott Berry: Hmm. 00:33:07.857 --> 00:33:08.097 Stephen Senn: but I 00:33:08.232 --> 00:33:08.242 Scott Berry: this, 00:33:08.307 --> 00:33:11.607 Stephen Senn: the organizational side is, is, uh, a great saving. 00:33:12.497 --> 00:33:12.787 Scott Berry: Yeah. 00:33:12.787 --> 00:33:13.067 Yeah. 00:33:13.077 --> 00:33:13.427 Right, 00:33:13.497 --> 00:33:14.787 Stephen Senn: during the COVID epidemic in 00:33:14.997 --> 00:33:15.347 Scott Berry: right. 00:33:15.957 --> 00:33:20.007 Stephen Senn: oneself to, drop an add arms and so forth quickly with a 00:33:20.007 --> 00:33:23.277 minimal amount of, uh, administrative fuss was, was very important. 00:33:24.942 --> 00:33:29.352 Scott Berry: So the, you touched on a larger topic sometime, uh, uh, scientific 00:33:29.352 --> 00:33:32.442 hype and actual reality a little bit. 00:33:32.442 --> 00:33:36.282 You, you talked about adaptive designs, but largely platform trials. 00:33:36.642 --> 00:33:42.042 There's been, the new FDA guidance is out, draft guidance on Bayesian statistics. 00:33:42.302 --> 00:33:47.712 ICH E20 draft was out, which talks about adaptive designs largely. 00:33:48.162 --> 00:33:53.472 Um, uh, it, do you have, uh, uh, what, what is your. 00:33:53.782 --> 00:33:58.797 Thoughts on all of that and the movement towards some Bayesian to adaptive designs. 00:33:59.182 --> 00:34:04.107 Stephen Senn: Well, I mean, decision analysis, um, teaches you, and I'm, I'm 00:34:04.107 --> 00:34:07.472 not denying it, that the, the option to change things is always valuable. 00:34:08.042 --> 00:34:08.392 So from 00:34:08.392 --> 00:34:11.652 that point of view, you can't, you can't argue against flexibility. 00:34:12.132 --> 00:34:16.692 Um, the option is not always as great as, uh, as some people think. 00:34:17.142 --> 00:34:20.382 I'm slightly annoyed about all of this because there are other simpler things 00:34:20.382 --> 00:34:23.502 that the FDA could have been doing a long, long time ago, which would've 00:34:23.502 --> 00:34:24.972 made a but much bigger difference. 00:34:25.392 --> 00:34:30.822 One of them would be banning dichotomies an extraordinary, extraordinary number. 00:34:31.182 --> 00:34:36.132 Of clinical trials still use information destroying dichotomies, and we know 00:34:36.132 --> 00:34:41.352 that as soon as you do that, on the best of cases, your sample size increases 00:34:41.352 --> 00:34:45.192 by about the necessary sample size increases by 50%, but it can easily 00:34:45.522 --> 00:34:49.242 double treble if you get the cut points wrong, if you get bad ones. 00:34:49.452 --> 00:34:50.792 So that's one particular point. 00:34:51.092 --> 00:34:54.642 The second thing was using covariates which one could have been using 00:34:54.642 --> 00:34:56.502 in a linear model since years ago. 00:34:56.772 --> 00:35:00.252 Now the FDA has gone up on some, uh, ridiculous. 00:35:00.642 --> 00:35:04.752 Uh, covariate hunt in terms of estimands and so forth. 00:35:05.022 --> 00:35:08.152 All of this is really of minor importance. 00:35:08.152 --> 00:35:11.922 The important was to use covariates to model, and we could have been doing 00:35:11.922 --> 00:35:14.052 that a long time ago and we weren't. 00:35:14.682 --> 00:35:14.902 Scott Berry: Hmm. 00:35:15.132 --> 00:35:19.242 Stephen Senn: I've even turned up a particular, um, meetings in which the 00:35:19.242 --> 00:35:22.782 head of a particular section, statistic section of the FDA said proudly. 00:35:22.932 --> 00:35:25.317 We don't do modeling and I think, gosh. 00:35:27.072 --> 00:35:32.082 How can a statistician say, say such a thing and be proud of it? 00:35:32.082 --> 00:35:33.132 It's unbelievable. 00:35:33.612 --> 00:35:34.092 So 00:35:34.932 --> 00:35:38.322 Scott Berry: I'm fully on board with you on both of those, those points. 00:35:38.327 --> 00:35:43.092 The, the, you know, the dichotomy is just, is, it's mind blowing that we do that. 00:35:43.362 --> 00:35:43.422 Yeah. 00:35:43.662 --> 00:35:44.592 Stephen Senn: Absolutely insane. 00:35:44.802 --> 00:35:48.402 You, you, you replace a, you replace the whole Kaplan Meier curve that 00:35:48.402 --> 00:35:50.442 you could have all that information. 00:35:50.712 --> 00:35:53.532 You replace it by just two points on the curve, you know, 00:35:53.592 --> 00:35:53.832 Scott Berry: Yeah. 00:35:54.837 --> 00:35:57.537 Stephen Senn: Uh, response rate or the death rate or whatever 00:35:57.537 --> 00:35:59.967 the rate is at two years rather 00:35:59.982 --> 00:36:00.312 Scott Berry: Right, 00:36:00.507 --> 00:36:03.627 Stephen Senn: having the whole, the whole thing that there is there, you know? 00:36:03.642 --> 00:36:04.992 Scott Berry: right, right, right. 00:36:05.202 --> 00:36:06.072 Uh, yep. 00:36:06.147 --> 00:36:09.117 Stephen Senn: as Kane said, in the long run, we're all dead, so, you know. 00:36:10.692 --> 00:36:10.962 Scott Berry: Yeah. 00:36:11.532 --> 00:36:11.982 Okay. 00:36:11.982 --> 00:36:15.462 So, uh, you, you had touched on something else that I, that, that I think is 00:36:15.462 --> 00:36:19.332 important and I wanna make sure it, it, it, you, you were able to talk about 00:36:19.332 --> 00:36:24.252 that is A-D-S-M-B in a more complicated trial within a platform, trial time 00:36:24.252 --> 00:36:29.047 adjustments going on, and the role of, uh, or the challenges of that. 00:36:29.712 --> 00:36:30.012 Stephen Senn: Yeah. 00:36:31.272 --> 00:36:31.482 Yeah. 00:36:31.482 --> 00:36:37.002 So, um, I mean, I think that's, that's challenging because, uh, what 00:36:37.002 --> 00:36:40.362 you're typically looking at is you're looking at all sorts of side effects. 00:36:42.282 --> 00:36:43.572 Potential side effects. 00:36:43.722 --> 00:36:46.602 I mean, let's call them, uh, adverse events, 00:36:46.642 --> 00:36:47.062 Scott Berry: Mm-hmm. 00:36:47.592 --> 00:36:50.022 Stephen Senn: without really knowing whether they're causal or not. 00:36:50.592 --> 00:36:56.202 Um, and, uh, it's very unlikely that you will have the machinery for doing the 00:36:56.202 --> 00:36:57.972 time adjustments for all these things. 00:36:57.972 --> 00:37:00.642 So you're actually having to make some sort of a judgment just by 00:37:00.642 --> 00:37:05.472 looking at raw data, where ideally you would want controlled, controlled 00:37:05.472 --> 00:37:08.652 data you'd like to be comparing, like with like, in order to do that. 00:37:09.402 --> 00:37:09.672 Scott Berry: Hmm. 00:37:10.167 --> 00:37:12.927 Stephen Senn: So, so this is one of the problems certainly. 00:37:13.587 --> 00:37:13.947 Scott Berry: Yeah. 00:37:14.037 --> 00:37:14.247 Yeah. 00:37:14.277 --> 00:37:19.017 So, and, and, and I do think it's a, in these more complicated trials where you 00:37:19.017 --> 00:37:24.957 have human oversight diving into what the model does know and doesn't know the model 00:37:24.957 --> 00:37:28.137 knows certain things and maybe there's a trust there, but there's lots of other 00:37:28.137 --> 00:37:33.417 things that it's critical to be able to do this well with the DSMB for sure. 00:37:33.782 --> 00:37:34.072 Stephen Senn: Yeah. 00:37:34.437 --> 00:37:34.917 Scott Berry: Um. 00:37:35.427 --> 00:37:39.147 You, you, you said something as we were, we were coming on, and I, I 00:37:39.147 --> 00:37:41.847 wonder, I think it'd be valuable for everybody to talk about, 00:37:41.847 --> 00:37:44.037 but you describe this evolution. 00:37:44.277 --> 00:37:47.907 I, and, and largely my question is to you, what, looking forward, 00:37:47.907 --> 00:37:51.867 what things do you think are important in clinical trial science? 00:37:52.227 --> 00:37:52.677 Um, you've seen. 00:37:53.652 --> 00:37:55.032 A good, uh, a good deal. 00:37:55.032 --> 00:37:58.302 And we talked about, you've been a statistician, you've been a professor, 00:37:58.542 --> 00:38:00.222 you've been at a pharmaceutical company. 00:38:00.222 --> 00:38:01.362 Lots of roles in this. 00:38:01.362 --> 00:38:05.082 So thinking about things going forward where we are, one of them you said, 00:38:05.082 --> 00:38:10.092 which I thought was really interesting, was the difference between eras in which 00:38:10.092 --> 00:38:15.672 we had private data and public analysis to public data and private analysis. 00:38:15.882 --> 00:38:18.972 I, I'd love for you to, to, to tell our listeners about that. 00:38:19.707 --> 00:38:23.922 Stephen Senn: Well, I suppose, um, even though I'm a frequentist, I mean. 00:38:24.627 --> 00:38:29.427 I usually do frequentist analysis and even though I sort of believe in the 00:38:29.427 --> 00:38:35.667 value of pre-specified analysis, I sort of wonder, well, know, especially when 00:38:35.667 --> 00:38:39.867 we looks at things like multiplicity, is it reasonable that just because a 00:38:39.867 --> 00:38:43.377 group of us got together on a particular day and we decided on this particularly 00:38:43.377 --> 00:38:48.627 complicated scheme for adjusting endpoints, that the whole of scientific 00:38:48.627 --> 00:38:52.527 posterity is now condemned to use this particular scheme that we chose. 00:38:54.087 --> 00:38:57.267 and, uh, of course the Bayesian answer would be, well, they're not, because 00:38:57.267 --> 00:38:58.977 people are not required to think alike. 00:38:59.397 --> 00:39:02.187 They start out with different, uh, prior distributions. 00:39:02.187 --> 00:39:03.507 They have different values. 00:39:03.957 --> 00:39:07.497 Um, and the, the sort of depressing result, end result of that is 00:39:07.497 --> 00:39:08.817 that we end up sharing data. 00:39:09.597 --> 00:39:13.947 Uh, there's no, uh, no analysis', still some value in, um. 00:39:15.447 --> 00:39:17.517 There's still some value in trust. 00:39:17.517 --> 00:39:21.837 I often say that you should think of the purposes of a protocol 00:39:21.837 --> 00:39:24.267 in terms of the five vowels. 00:39:24.717 --> 00:39:27.837 A for anticipation, it's your thought experiment. 00:39:28.527 --> 00:39:29.457 E for ethics. 00:39:29.457 --> 00:39:32.127 It's the way in which you explore the ethical problems 00:39:32.127 --> 00:39:33.267 that could arise with the trial. 00:39:33.327 --> 00:39:38.157 I, for inference, which is what you and I are interested in, O for organization and 00:39:38.157 --> 00:39:41.007 you for utmost good faith is utmost good. 00:39:41.007 --> 00:39:42.087 Faith was important. 00:39:42.087 --> 00:39:43.377 If we're gonna share data. 00:39:44.097 --> 00:39:47.697 we have to know how did the data arise? 00:39:49.227 --> 00:39:52.617 What was done before we saw them, which 00:39:52.857 --> 00:39:53.277 Scott Berry: Mm-hmm. 00:39:54.147 --> 00:39:54.437 Yeah. 00:39:54.907 --> 00:39:55.197 Yeah. 00:39:55.227 --> 00:39:59.007 Stephen Senn: So I, I think that we're moving towards a, an era in which the 00:39:59.007 --> 00:40:00.867 data will be available on the web. 00:40:01.737 --> 00:40:04.077 Um, we're gonna have all sorts of problems with anonymizing. 00:40:05.022 --> 00:40:10.092 But then we will have, uh, to uh, sort of adapt to quotation of, uh, chairman 00:40:10.092 --> 00:40:13.212 Mao's, let a thousand analyses flourish. 00:40:13.302 --> 00:40:18.222 You know, so we we're gonna see lots and lots of different analyses, uh, 00:40:18.222 --> 00:40:23.202 and the, uh, the problem of, uh, multiplicity will enter a new world. 00:40:23.617 --> 00:40:23.697 Scott Berry: Hmm. 00:40:24.252 --> 00:40:27.312 Stephen Senn: and maybe it's not so quite much hidden data. 00:40:27.312 --> 00:40:29.532 It's, uh, analysis missing, not at random. 00:40:29.532 --> 00:40:33.762 We should worry about, you know, on only the interesting ones will be reported. 00:40:34.527 --> 00:40:37.527 The, the dozens and dozens of boring ones will not make it. 00:40:38.007 --> 00:40:38.367 Scott Berry: Hmm. 00:40:38.517 --> 00:40:40.527 Stephen Senn: I, I think that's a, that's an issue 00:40:41.487 --> 00:40:41.667 Scott Berry: Yeah. 00:40:42.087 --> 00:40:42.957 Yeah, I, I 00:40:43.227 --> 00:40:43.407 agree. 00:40:43.467 --> 00:40:43.677 Stephen Senn: that. 00:40:43.677 --> 00:40:50.937 I think is also that, that the idea of the evidence from a study, which 00:40:50.937 --> 00:40:55.197 was always rather suspect from a Bayesian point of view, because it 00:40:55.197 --> 00:40:59.757 would depend on your prior distribution as to how evidential the study was, 00:41:00.657 --> 00:41:03.657 um, is also coming under scrutiny. 00:41:05.037 --> 00:41:05.277 Scott Berry: Hmm. 00:41:05.292 --> 00:41:07.182 Stephen Senn: you can see that with your adaptive design. 00:41:07.182 --> 00:41:10.932 If you have a look, you'll see that the information continues to 00:41:10.932 --> 00:41:16.812 accrue for treatment, number one, even though it's been abandoned 00:41:17.487 --> 00:41:17.667 Scott Berry: Hmm. 00:41:19.002 --> 00:41:19.272 Stephen Senn: in, 00:41:19.407 --> 00:41:19.767 Scott Berry: Yeah. 00:41:19.992 --> 00:41:20.772 Stephen Senn: design you're looking at. 00:41:20.772 --> 00:41:24.432 Because what's happening is we continue to have control. 00:41:24.432 --> 00:41:28.812 So although we learn nothing more directly about treat number one, indirectly, we 00:41:28.812 --> 00:41:30.642 learn something about it because of the 00:41:30.867 --> 00:41:31.047 Scott Berry: Yeah. 00:41:31.047 --> 00:41:31.857 And then. 00:41:31.992 --> 00:41:32.862 Stephen Senn: no fixed. 00:41:33.327 --> 00:41:35.012 Evidence from a particular study. 00:41:35.122 --> 00:41:36.332 It's all relative. 00:41:36.987 --> 00:41:37.317 Scott Berry: Yeah. 00:41:37.492 --> 00:41:41.877 And, and that actually was absolutely in the I SPY two trial over 10 years. 00:41:41.877 --> 00:41:46.827 We had 27 arms and the inferences about arm one that was in 00:41:46.827 --> 00:41:48.447 there continued to change. 00:41:48.447 --> 00:41:52.917 Now it was very, very small, but it did continue to change as the, as the 00:41:52.917 --> 00:41:54.387 data accrued during the course of that. 00:41:55.257 --> 00:41:55.587 Yeah. 00:41:56.082 --> 00:41:56.442 Right. 00:41:56.592 --> 00:41:57.732 Right, right, right. 00:41:58.092 --> 00:42:01.392 Um, I it, do you find value? 00:42:01.392 --> 00:42:03.702 I, um, a question outta nowhere. 00:42:03.732 --> 00:42:07.242 Uh, uh, do you find value since you, you rail against 00:42:07.242 --> 00:42:09.342 the dichotomizing of endpoints? 00:42:09.942 --> 00:42:13.182 Do you have a similar frustration with the dichotomizing of a 00:42:13.182 --> 00:42:14.982 trial as success or failure? 00:42:15.192 --> 00:42:20.502 And does a Bayesian play a role in quantifying evidence, perhaps above 00:42:20.502 --> 00:42:22.422 and beyond frequentist in those trials? 00:42:24.497 --> 00:42:26.427 Stephen Senn: Well, I think that, um. 00:42:27.747 --> 00:42:32.187 There's a sense in which, at the point at which you have to make 00:42:32.187 --> 00:42:35.397 a decision, things are binary. 00:42:36.417 --> 00:42:40.887 So you do have to make a decision for a given patient as to whether 00:42:40.887 --> 00:42:42.417 to use one treatment or another. 00:42:43.317 --> 00:42:50.397 Um, in theory, what you could say is, well, we're going to delegate the 00:42:50.397 --> 00:42:52.917 decision making away from the FDA. 00:42:54.027 --> 00:42:58.257 Um, what the FDA will do instead is the FDA will say that, uh, 00:42:58.677 --> 00:43:00.747 these data have a seal of approval. 00:43:01.707 --> 00:43:05.277 They are data that you can use to make your own decision. 00:43:05.547 --> 00:43:08.547 Now it's over to you, the doctor and the patient to make the decision. 00:43:09.537 --> 00:43:14.187 And a long time ago, Jurgen Hilden, uh, sort of, uh, very good Danish 00:43:14.187 --> 00:43:17.637 statistician who was interested in utility theory, he proposed this. 00:43:17.637 --> 00:43:20.307 He actually said that what you should do is you should produce. 00:43:20.922 --> 00:43:24.312 Um, an analysis of all the various outcomes, all the things that might 00:43:24.312 --> 00:43:28.512 matter to a patient, and then every patient could look at them together and 00:43:28.512 --> 00:43:30.162 they could do the trade off themselves. 00:43:30.252 --> 00:43:34.332 They could decide whether they would, so eventually there would 00:43:34.332 --> 00:43:36.222 have to be a binary decision. 00:43:36.222 --> 00:43:40.782 The patient's gonna have to decide to take Tri Pill A or pill B or something else. 00:43:40.967 --> 00:43:41.187 Scott Berry: Hmm. 00:43:41.262 --> 00:43:45.732 Stephen Senn: But, it doesn't mean that you have to think of 00:43:45.732 --> 00:43:46.837 a trial in that particular way. 00:43:48.747 --> 00:43:49.707 Provide information. 00:43:50.787 --> 00:43:56.067 I can see all sorts of difficulties in making this, uh, a way in which 00:43:56.067 --> 00:44:00.237 society will behave, but I wouldn't necessarily argue against it. 00:44:00.237 --> 00:44:00.837 I think, you know, 00:44:01.122 --> 00:44:01.362 Scott Berry: Hmm. 00:44:02.217 --> 00:44:04.557 Stephen Senn: I think it's also, by the way, I think relates to 00:44:04.557 --> 00:44:08.007 this is a misunderstanding about, um, clinic irrelevant differences. 00:44:08.012 --> 00:44:09.592 Delta Delta, um, 00:44:10.092 --> 00:44:10.422 Scott Berry: Okay. 00:44:10.722 --> 00:44:10.932 Yeah. 00:44:11.247 --> 00:44:13.872 Stephen Senn: delta for me is not what you expect the drug to do. 00:44:15.222 --> 00:44:19.662 It's essentially some way in which you scale the information, because what 00:44:19.662 --> 00:44:23.712 you want in a trial is you want the trial to provide a valuable amount of 00:44:23.712 --> 00:44:30.492 information, and that means that the, um, data precision, essentially the 00:44:30.492 --> 00:44:35.472 standard error divided by some function of n uh, so standard deviation divided 00:44:35.472 --> 00:44:39.492 by some function, square root function of n or whatever, that that should be 00:44:39.492 --> 00:44:43.602 some multiple of what you consider an important quantum of information to be. 00:44:43.737 --> 00:44:43.917 Scott Berry: Hmm, 00:44:44.172 --> 00:44:46.512 Stephen Senn: It's that particular ratio that you're targeting and 00:44:46.512 --> 00:44:50.202 the clinical relevant differences, a sort of way of, scaling that. 00:44:50.757 --> 00:44:51.057 Scott Berry: Hmm. 00:44:51.492 --> 00:44:53.562 Stephen Senn: so yeah, I think trials shouldn't, they're not failures 00:44:53.562 --> 00:44:56.532 or successes that trials provide a certain amount of information. 00:44:56.532 --> 00:44:57.432 And then, you know, 00:45:00.567 --> 00:45:01.077 Scott Berry: Awesome. 00:45:01.557 --> 00:45:02.127 Awesome. 00:45:02.277 --> 00:45:06.027 So, uh, uh, any, any other closing comments? 00:45:06.882 --> 00:45:07.542 Steven, 00:45:07.872 --> 00:45:09.162 Stephen Senn: uh, no. 00:45:09.162 --> 00:45:10.557 Apart from getting my best regards to Don. 00:45:11.352 --> 00:45:11.947 Scott Berry: I will. 00:45:12.357 --> 00:45:16.347 Stephen Senn: uh, no, I don't, I don't think so. 00:45:16.827 --> 00:45:18.027 I mean, um, 00:45:20.397 --> 00:45:24.477 apart from, I would say that, um, people should 00:45:26.517 --> 00:45:28.527 think about concurrent control. 00:45:29.667 --> 00:45:33.507 it's not the be all and the end all, but it does all sorts of things for you. 00:45:33.822 --> 00:45:34.032 Scott Berry: Hmm. 00:45:34.167 --> 00:45:40.647 Stephen Senn: also, um, blinding, if you can run a double-blind Randomized trial 00:45:40.647 --> 00:45:42.387 that also cures all sorts of things. 00:45:42.387 --> 00:45:46.017 If you're not careful, you're liable to overlook things, which 00:45:46.017 --> 00:45:50.897 would be impossible if the trial is randomized and double-blind Hmm. 00:45:51.027 --> 00:45:56.547 for instance, in a trial of a vaccine, you might say, well, you know, the 00:45:56.547 --> 00:46:00.087 people we're going to vaccinate, they can come to the center to be vaccinated. 00:46:00.357 --> 00:46:02.877 We can't run the trial double-blind so there's no point 00:46:02.877 --> 00:46:04.437 calling the control people in. 00:46:05.232 --> 00:46:08.262 We'll get nurses to go and take blood samples from them to 00:46:08.262 --> 00:46:10.122 see if they're seropositive or negative. 00:46:10.482 --> 00:46:13.122 And already what you find is that now the blood samples 00:46:13.122 --> 00:46:14.202 are being handled differently 00:46:14.622 --> 00:46:14.702 Hmm. 00:46:14.952 --> 00:46:17.532 and maybe they're being sent off at a different time in a different lab. 00:46:17.532 --> 00:46:20.952 If it's the same lab, then we know that assays vary over time, and 00:46:20.952 --> 00:46:24.972 so without really realizing it, the measurement process itself is 00:46:24.972 --> 00:46:29.342 introduced to bias simply because not randomized, double-blind If it's 00:46:29.342 --> 00:46:31.212 randomized, double-blind it's impossible. 00:46:31.212 --> 00:46:34.032 There is no way that you can correlate. 00:46:34.812 --> 00:46:39.912 taking of any sort of measurement with the allocation of either treatment because 00:46:40.152 --> 00:46:42.012 it's random and nobody knows what the 00:46:42.387 --> 00:46:42.627 Scott Berry: Hmm. 00:46:43.497 --> 00:46:43.737 Hmm. 00:46:45.387 --> 00:46:45.897 Agree. 00:46:46.197 --> 00:46:46.797 Agree. 00:46:47.097 --> 00:46:48.417 Well, thank you so much. 00:46:48.417 --> 00:46:49.977 Thank you for your blogs. 00:46:50.067 --> 00:46:50.772 Thank you for your. 00:46:51.132 --> 00:46:51.372 Stephen Senn: Okay. 00:46:52.002 --> 00:46:52.332 Scott Berry: Yep. 00:46:52.512 --> 00:46:56.412 Thank you for the pictures of your hikes and enjoy your hike tomorrow. 00:46:56.772 --> 00:47:00.702 Uh, I know you've been in a lot of interims, but thanks for joining 00:47:00.702 --> 00:47:03.882 us, uh, here and for everybody. 00:47:03.972 --> 00:47:07.122 Uh, till next time, we'll be here in the interim. 00:47:07.542 --> 00:47:07.992 Stephen Senn: Okay. 00:47:08.292 --> 00:47:08.652 Yeah. 00:47:08.802 --> 00:47:09.282 Thanks. 00:47:09.462 --> 00:47:09.702 Bye. 00:47:10.362 --> 00:47:11.022 Scott Berry: Thank you.