WEBVTT

NOTE
This file was generated by Descript 

00:00:00.029 --> 00:00:03.299
Judith: Welcome to Berry's In the
Interim podcast, where we explore the

00:00:03.299 --> 00:00:06.989
cutting edge of innovative clinical
trial design for the pharmaceutical and

00:00:06.989 --> 00:00:09.340
medical industries, and so much more.

00:00:09.659 --> 00:00:10.470
Let's dive in.

00:00:11.478 --> 00:00:14.778
Scott Berry: Well, welcome back
everyone to in the interim,

00:00:15.258 --> 00:00:16.668
I'm your host, Scott Perry.

00:00:17.838 --> 00:00:19.038
So welcome back.

00:00:19.068 --> 00:00:23.358
Um, for those of you who have joined
multiple times, I hope you still have

00:00:23.358 --> 00:00:25.458
enough alpha to keep joining us here.

00:00:25.458 --> 00:00:30.588
In the interim, uh, I of course make
jokes, but that is related to today's

00:00:30.588 --> 00:00:32.958
topic, so I'm going solo today.

00:00:34.488 --> 00:00:40.578
And by, oh, by the way, I've had people,
uh, if, if you can see the video, uh,

00:00:40.578 --> 00:00:47.568
of me, you can see in my background, I
have this beautiful, uh, view of trees.

00:00:47.598 --> 00:00:49.758
It looks like I'm in a tree house.

00:00:50.268 --> 00:00:52.758
I'm actually at my cabin.

00:00:52.758 --> 00:00:56.568
So we have a cabin in
near Brainerd, Minnesota.

00:00:57.153 --> 00:01:01.833
So I grew up in Minnesota and, uh, many
people in Minnesota have their little

00:01:01.833 --> 00:01:07.803
cabin on a lake somewhere, and we spend
about three months, uh, somewhere three

00:01:07.803 --> 00:01:09.423
to four months up here in Minnesota.

00:01:09.423 --> 00:01:12.003
We get outta the heat of Austin, Texas.

00:01:12.003 --> 00:01:12.783
In the summer.

00:01:13.113 --> 00:01:18.633
We get to enjoy this beautiful,
uh, Minnesota summer, uh, without,

00:01:18.813 --> 00:01:22.443
without having the Minnesota
winter or the Austin summer.

00:01:22.443 --> 00:01:24.753
We kind of get the bo best of both.

00:01:25.788 --> 00:01:28.908
We are, we are approaching the
time at the end of the year

00:01:28.908 --> 00:01:30.798
that we travel back to Austin.

00:01:30.858 --> 00:01:31.968
It'll be in a couple weeks.

00:01:31.968 --> 00:01:33.288
We travel back to Austin.

00:01:33.288 --> 00:01:37.278
So you, you won't see this
view of my, my tree house.

00:01:37.278 --> 00:01:41.688
I'm in the, essentially in a little
room above the garage here in the cabin,

00:01:42.168 --> 00:01:44.628
uh, where I do my work in the podcast.

00:01:44.628 --> 00:01:48.738
And we have this beautiful, beautiful
set of trees, uh, out the window.

00:01:48.738 --> 00:01:49.578
It's gorgeous.

00:01:50.973 --> 00:01:56.463
We're about to embark on this trip
back from Minnesota to Austin, and

00:01:56.463 --> 00:02:02.283
we have a big dog, uh, a Bernice
Mountain dog poodle mix, a bernedoodle.

00:02:02.283 --> 00:02:06.783
He is about 110 pounds, so of
course we drive back and forth

00:02:06.783 --> 00:02:08.463
each way, so we drive up here.

00:02:09.183 --> 00:02:15.213
We drive back, I've even started to
two days is just kind of, uh, brutal.

00:02:15.273 --> 00:02:18.873
Uh, it's about 20 hours, 19 hours total.

00:02:19.263 --> 00:02:21.483
So I've started to try
to do that in one day.

00:02:21.573 --> 00:02:26.973
Uh, my wife won't do it with me, but,
um, um, the dog and I jump in the car

00:02:26.973 --> 00:02:30.783
and we leave at four and we get in,
you know, 10, 11 o'clock at night.

00:02:31.623 --> 00:02:32.793
I think we're headed back.

00:02:32.793 --> 00:02:37.353
I think we're gonna do it two days,
but I wanna sort of set up that when

00:02:37.353 --> 00:02:41.763
we do this, there's multiple cities
that we drive through, so we're

00:02:41.763 --> 00:02:43.503
a couple hours from Minneapolis.

00:02:43.503 --> 00:02:46.533
There's multiple ways in
and around Minneapolis.

00:02:46.798 --> 00:02:48.688
And the traffic in Minneapolis St.

00:02:48.688 --> 00:02:50.998
Paul has been, uh, has been challenging.

00:02:51.058 --> 00:02:56.338
It's it's construction season, uh, as
every summer is here in, in Minnesota.

00:02:56.698 --> 00:02:58.558
We drive through Des Moines.

00:02:58.558 --> 00:03:03.118
That's usually not too bad, but Kansas
City has multiple routes around it that

00:03:03.118 --> 00:03:05.188
you can take depending on the traffic.

00:03:05.668 --> 00:03:08.488
Same with, uh, Oklahoma City.

00:03:08.488 --> 00:03:11.758
And the, the, the, the biggest
hurdle is Dallas-Fort Worth.

00:03:12.128 --> 00:03:13.868
Depending on the timing, we go through it.

00:03:13.868 --> 00:03:16.928
And there are many ways in
and around Dallas-Fort Worth,

00:03:17.198 --> 00:03:18.608
depending on the traffic.

00:03:19.958 --> 00:03:23.678
So we get ready to drive out
and we're thinking there's

00:03:23.678 --> 00:03:25.358
multiple routes we can go.

00:03:25.358 --> 00:03:29.378
And of course, we're aided
by our, our, uh, maps.

00:03:29.648 --> 00:03:34.688
Um, and we can get updated, uh,
views of, of traffic and, and,

00:03:34.688 --> 00:03:36.038
and how long each one's gonna go.

00:03:37.023 --> 00:03:42.273
So do I decide on how I'm gonna get
around Minneapolis when I drive out

00:03:42.273 --> 00:03:47.313
of our cabin here in Brainerd, I'm
still, you know, two hours plus away.

00:03:47.343 --> 00:03:53.703
Do I decide how I'm going around Dallas
Fort Worth when I go through Oklahoma

00:03:53.703 --> 00:03:56.073
City, where I'm still a couple hours?

00:03:57.558 --> 00:04:02.478
It seems like that would be absolutely
nuts to make that decision, especially

00:04:02.868 --> 00:04:09.078
when I have an app that gives me
the most current view of traffic.

00:04:09.258 --> 00:04:10.878
I'm gonna make that decision.

00:04:11.658 --> 00:04:16.008
When I need to make that decision as
close as I can to making that decision,

00:04:16.338 --> 00:04:22.308
and I'm gonna, it's gonna provide me the
most efficient way home, uh, which, which

00:04:22.308 --> 00:04:24.618
for me is the shortest route in that.

00:04:25.068 --> 00:04:31.608
And forcing myself to make that decision
two hours earlier or, you know, five

00:04:31.728 --> 00:04:36.558
hours earlier, uh, seems to be bizarre.

00:04:38.088 --> 00:04:38.508
Okay.

00:04:39.213 --> 00:04:41.643
So let's jump in.

00:04:41.643 --> 00:04:46.473
Today's topic and today's topic
is the promising zone design.

00:04:47.373 --> 00:04:51.213
It is, you know, here at Berry
Consultants we're in the interim.

00:04:51.663 --> 00:04:54.423
We, we, we do adaptive designs.

00:04:54.423 --> 00:04:56.343
It's, it's really our expertise.

00:04:56.673 --> 00:05:01.443
Uh, I've been doing this for 25 years and
the promising zone is an adaptive design.

00:05:01.803 --> 00:05:03.933
It's an adaptive sample size approach.

00:05:04.383 --> 00:05:06.543
So let's talk about the promising zone.

00:05:07.323 --> 00:05:12.363
Uh, it's actually a beautiful,
neat mathematical result.

00:05:12.993 --> 00:05:16.203
The question, is that mathematical
result tied to a design?

00:05:16.203 --> 00:05:16.863
Is it good?

00:05:16.893 --> 00:05:17.853
Does it work well?

00:05:19.263 --> 00:05:20.733
So let's get into that.

00:05:20.733 --> 00:05:25.983
So let's, and one of the challenges,
and it's sort of fun in doing this

00:05:25.983 --> 00:05:28.323
podcast, is I can't use slots.

00:05:29.238 --> 00:05:31.758
Now I think there's probably a
functionality where I can put

00:05:31.758 --> 00:05:35.448
slides out there and all that, but
you don't want, you don't want a,

00:05:35.928 --> 00:05:38.478
um, uh, slides and all of that.

00:05:38.478 --> 00:05:40.998
So some of you are out for a jog.

00:05:40.998 --> 00:05:43.308
Some of you are driving
to work in the morning.

00:05:43.548 --> 00:05:46.698
You're consuming this in
different ways, and so I sort

00:05:46.698 --> 00:05:48.048
of have to describe this to you.

00:05:48.048 --> 00:05:51.138
So I'm gonna walk through a little
bit of this with, with no slide.

00:05:52.443 --> 00:05:56.343
Uh, which is, which is a
challenge, but um, uh, I think

00:05:56.343 --> 00:05:57.513
is a good challenge actually.

00:05:57.513 --> 00:06:01.683
And something to think about when you
give a talk at a conference is can

00:06:01.683 --> 00:06:06.003
you do that presentation where slides
are an aid to you but not a crutch?

00:06:06.273 --> 00:06:07.203
So I'll try to do that.

00:06:07.263 --> 00:06:07.953
I may fail.

00:06:08.643 --> 00:06:14.163
So let's the simplest of designs
where this is adaptive sample size.

00:06:14.163 --> 00:06:15.843
We're running a phase three trial.

00:06:15.888 --> 00:06:16.098
Wow.

00:06:17.238 --> 00:06:21.528
We have a standard one-sided 0.025

00:06:21.858 --> 00:06:26.418
and, uh, you know, I've said this
on multiple podcasts, but, uh,

00:06:26.418 --> 00:06:31.248
a two-sided superiority trial
is just strange, uh, in this.

00:06:31.248 --> 00:06:34.668
So we'll continue to talk
about one sided alpha of 0.025.

00:06:36.558 --> 00:06:42.288
Now, um, I'm gonna set this up within
the context of a test of a mean.

00:06:42.498 --> 00:06:47.358
So we're doing a, at the end of the
day, we're doing a T-test in Ancova,

00:06:47.718 --> 00:06:52.638
an MMRM, where we're doing a test
of means and a higher mean is good.

00:06:53.753 --> 00:06:56.918
We will think of a standard
deviation of 12, just so I can

00:06:56.918 --> 00:06:58.718
have a range of effect sizes.

00:06:59.018 --> 00:07:04.178
And in this scenario, let's sort
of identify that a mean change

00:07:04.178 --> 00:07:09.668
of four points on this endpoint
is really where I'm targeting.

00:07:09.668 --> 00:07:13.778
I, I, I think in my phase two trials
that we might be as good as five.

00:07:13.778 --> 00:07:17.408
We may even be north of that,
but I, I really want to make

00:07:17.408 --> 00:07:18.668
sure I'm powered for four.

00:07:19.243 --> 00:07:22.543
You know, three is clinically
reasonable in this setting.

00:07:22.543 --> 00:07:25.783
If I, if I can do three,
that's a marketable drug

00:07:25.783 --> 00:07:27.223
that's gonna help patients.

00:07:27.498 --> 00:07:31.183
I, I think it's approvable, but it's
a big trial, five to power three.

00:07:31.183 --> 00:07:32.053
It's a bigger trial.

00:07:32.083 --> 00:07:35.923
And so that, that's where I'm stuck
with, and a lot of companies are stuck

00:07:35.923 --> 00:07:37.483
with this sort of exact scenario.

00:07:37.663 --> 00:07:41.593
I, I might be five, I might be standing
on a drug that has a mean change of

00:07:41.593 --> 00:07:44.083
five in my population, enrolling.

00:07:44.263 --> 00:07:46.723
So one-to-one trial, straightforward.

00:07:48.243 --> 00:07:52.833
So in that, uh, a, a sta a fairly
standard thing is I can think of okay,

00:07:52.833 --> 00:07:57.603
that that affects size of four to
be powered for four with a standard

00:07:57.603 --> 00:08:00.813
deviation of 12, one to one randomized.

00:08:00.813 --> 00:08:05.283
I'm slightly over 80%
powered if I do 300 patients.

00:08:05.468 --> 00:08:11.258
150 and 150, slightly over 80%
powered, very standard decision

00:08:11.888 --> 00:08:17.168
that a sponsor can make to run a
fixed sample size 300 patient trial.

00:08:18.488 --> 00:08:22.208
Now the question is, can, can we do
better that than that with an adaptive

00:08:22.208 --> 00:08:26.768
sample size if we did a 500 patient trial?

00:08:27.213 --> 00:08:31.023
I'd be 80% powered for
the effect of three.

00:08:31.023 --> 00:08:37.113
That slightly smaller effect if I run
500, but I don't wanna run a 500 patient

00:08:37.113 --> 00:08:39.333
trial because I think I'm north of that.

00:08:39.363 --> 00:08:42.993
But if it's three, yes, I, I, I wish
I would've run a 500 patient trial.

00:08:43.863 --> 00:08:50.163
Now, I could run a 200 patient
trial and I'd be about 84% powered

00:08:50.553 --> 00:08:52.413
for an effect size of five.

00:08:53.463 --> 00:08:56.283
So I, I could, I could run
a trial anywhere in there.

00:08:56.643 --> 00:08:58.893
200, 300, 500.

00:08:59.103 --> 00:09:01.113
So can we do an adaptive sample size?

00:09:01.113 --> 00:09:05.643
And maybe this is the perfect
scenario for a promising zone design.

00:09:06.723 --> 00:09:10.983
Uh, and what we'll come back to sort
of different ways to view this, but

00:09:10.983 --> 00:09:13.443
what is the promising zone design?

00:09:14.013 --> 00:09:19.113
This was, uh, na ap aptly named
and we'll come back to, to, um,

00:09:19.203 --> 00:09:21.813
uh, the name Promising Zone Design.

00:09:21.813 --> 00:09:23.703
It's, it's, it's a beautiful name.

00:09:24.693 --> 00:09:28.203
But it's a beautiful
mathematical result as well.

00:09:28.623 --> 00:09:37.383
So, uh, meta and Pocock 2000, uh, 10
article in Statistics in Medicine.

00:09:37.533 --> 00:09:38.103
Name.

00:09:38.103 --> 00:09:42.513
The design that uses this and the
beautiful, clever mathematical

00:09:42.783 --> 00:09:48.543
trick here is they, they have a way
in which they don't spend alpha.

00:09:49.443 --> 00:09:53.283
And it's a really neat, uh,
it's a powerful result to think

00:09:53.283 --> 00:09:54.723
about in adaptive designs.

00:09:55.413 --> 00:09:57.303
So what, what is the result?

00:09:58.263 --> 00:10:03.213
The result is, and I want you to
envision a sort of two by two table

00:10:03.213 --> 00:10:08.163
and it exemplifies that looking
at data doesn't cause alpha.

00:10:09.003 --> 00:10:15.663
It's the actions you take,
depending on the data that can, can

00:10:15.663 --> 00:10:18.033
trigger the need to adjust Alpha.

00:10:19.683 --> 00:10:22.803
And that sort of two
by two table is on one.

00:10:22.803 --> 00:10:27.483
The columns here is that the data
are good or the data are bad.

00:10:27.543 --> 00:10:30.093
And I'll come back to what,
what exactly that means.

00:10:30.663 --> 00:10:35.133
But think about good data at an
interim and bad data at an interim.

00:10:35.883 --> 00:10:42.423
And then the different sort of actions
you can take are making the trial

00:10:42.423 --> 00:10:45.873
smaller and making the trial bigger.

00:10:46.653 --> 00:10:50.103
So that's my two by two tape
matrix I'm looking at here.

00:10:51.723 --> 00:10:57.498
When the data are bad and you make the
trial smaller, usually that's futility.

00:10:59.133 --> 00:11:03.783
So I, I do an interim
and the data are bad.

00:11:03.843 --> 00:11:07.023
I'll come back to bad and good, and
that's the sort of clever trick of

00:11:07.023 --> 00:11:11.733
the promising zone design is when
the data are bad and I make the trial

00:11:11.733 --> 00:11:13.863
smaller, I typically stop for futility.

00:11:14.193 --> 00:11:14.913
You, you lower.

00:11:15.883 --> 00:11:20.563
The overall probability of a type one
error in the trial because largely you

00:11:20.563 --> 00:11:23.023
can't win that trial in that scenario.

00:11:23.023 --> 00:11:27.643
So things that lessen the sample
size in that case are going

00:11:27.643 --> 00:11:29.113
to deflate type one error.

00:11:29.113 --> 00:11:31.153
Nothing you do in that realm.

00:11:31.423 --> 00:11:34.693
Inflates type one error, looking
at the data and doing futility

00:11:34.693 --> 00:11:36.133
decreases type one error.

00:11:37.843 --> 00:11:43.093
Now if you make the trial smaller,
when data are good, this is

00:11:43.093 --> 00:11:44.623
really sort of group sequential.

00:11:45.618 --> 00:11:50.178
If the data are strong enough and we
stop the trial, what people refer to

00:11:50.178 --> 00:11:55.218
as early, and I think the terminology
is you, you, you declare superiority.

00:11:56.058 --> 00:12:00.108
That's the opposite sort of action
you take when the, when the data

00:12:00.108 --> 00:12:04.878
are good, making it smaller,
that can trigger inflating type

00:12:04.878 --> 00:12:06.468
one error without adjustment.

00:12:06.768 --> 00:12:10.938
So we, when we're gonna do that,
we're, we're gonna make it smaller.

00:12:11.988 --> 00:12:14.268
When the data are good, we adjust alpha.

00:12:15.888 --> 00:12:18.528
So that's, that's making a trial smaller.

00:12:18.678 --> 00:12:21.888
In some cases you don't
inflate type one error.

00:12:21.888 --> 00:12:23.838
In some cases you do
inflate type one error.

00:12:25.278 --> 00:12:33.018
When, um, uh, when I make a trial
bigger, that's, that's somewhat

00:12:33.078 --> 00:12:34.458
the opposite of that group.

00:12:34.458 --> 00:12:38.148
Sequential type triggering
calling superiority or futility.

00:12:38.778 --> 00:12:41.628
So suppose the data are good.

00:12:42.708 --> 00:12:43.728
Uh, let's do the other one.

00:12:43.788 --> 00:12:50.718
Suppose the data are bad and
I make the trial bigger, that

00:12:50.718 --> 00:12:52.788
actually inflates type one error.

00:12:54.408 --> 00:12:55.698
And it's the circumstance.

00:12:55.698 --> 00:13:01.728
If I was originally planning a trial
of 300 and my data are bad and I

00:13:01.728 --> 00:13:03.318
say, you know, let's go to 500.

00:13:04.488 --> 00:13:08.028
That is something where
you're, you're, you're, you're.

00:13:08.748 --> 00:13:12.618
Taking a bigger sample size
because the data are bad.

00:13:12.618 --> 00:13:16.248
You're trying to wash out the, the
negative data you have now with a bigger

00:13:16.248 --> 00:13:18.678
sample size, that inflates type one error.

00:13:20.208 --> 00:13:27.558
Now, when the data are good and
I make the trial bigger, you

00:13:27.558 --> 00:13:29.268
do not inflate type one error.

00:13:30.618 --> 00:13:34.608
Now, one of the actions that
fall into this bucket is

00:13:34.608 --> 00:13:36.618
response adaptive randomization.

00:13:38.283 --> 00:13:44.313
When you have multiple arms in a trial
and you increase the randomization to

00:13:44.313 --> 00:13:50.163
an arm that's doing better, you have
an effective sample size that's bigger.

00:13:51.483 --> 00:13:52.233
You're you.

00:13:52.263 --> 00:13:55.773
When data are good, you're giving
it, we want more data on that arm.

00:13:55.893 --> 00:13:57.543
You actually decrease type one error.

00:13:58.428 --> 00:14:02.598
Now that's a, that's, that's a
multi-arm trial scenario here.

00:14:02.898 --> 00:14:06.498
And in this case, uh,
that's not a two arm trial.

00:14:06.498 --> 00:14:07.758
That's a multiple arm scenario.

00:14:07.968 --> 00:14:12.498
But the other part where this happens,
where the data are good and you make

00:14:12.498 --> 00:14:14.628
the trial bigger is the promising zone.

00:14:16.128 --> 00:14:20.658
Now that's the neat little trick
about the promising zone is when the

00:14:20.658 --> 00:14:24.858
data are good and you make the trial
bigger, you don't need to adjust alpha.

00:14:26.448 --> 00:14:30.168
Now I said, what does good and bad mean?

00:14:30.798 --> 00:14:32.058
It's a neat result.

00:14:32.058 --> 00:14:36.408
Where is, suppose I'm going
after a trial that's 300 patients

00:14:36.408 --> 00:14:38.598
and I look at the data at 200.

00:14:40.548 --> 00:14:42.348
What does good and bad mean?

00:14:42.588 --> 00:14:48.348
Largely, if I am trending to a
successful trial at 300, meaning

00:14:48.348 --> 00:14:51.408
my conditional power is above 0.5,

00:14:51.858 --> 00:14:55.818
that if I do exactly where I am or
a little bit better, so on my effect

00:14:55.818 --> 00:15:01.158
size is what I need to win or bigger,
and I increase the sample size.

00:15:01.338 --> 00:15:03.288
There's no need to adjust alpha.

00:15:04.398 --> 00:15:08.928
Because I'm in this realm of my data
are good and I make the trial bigger.

00:15:09.288 --> 00:15:11.358
It's kind of the opposite
of group sequential.

00:15:11.478 --> 00:15:13.698
When the data are good,
you make the trial smaller.

00:15:13.698 --> 00:15:14.313
It needs adjustment.

00:15:15.468 --> 00:15:19.878
So that line of good and bad
is a 50% conditional power.

00:15:20.178 --> 00:15:24.678
Now you can, you can lower that a
little bit and the paper talks a lot

00:15:24.678 --> 00:15:29.088
about defining good and bad depending
on when you look and when the 300 is.

00:15:29.358 --> 00:15:33.918
So they give an example where they
can do a cutoff at 37% and that

00:15:33.918 --> 00:15:38.058
defines good and bad and no, no
adjustment to alpha as needed.

00:15:38.358 --> 00:15:40.248
As long as we're 0.37

00:15:40.248 --> 00:15:43.908
or better, we could increase 300 to 500.

00:15:44.463 --> 00:15:45.603
And you don't adjust.

00:15:47.048 --> 00:15:52.503
Alpha's really neat sort of, uh, summary
of adaptive designs and spend of alpha.

00:15:52.503 --> 00:15:54.543
It's a really neat mathematical result.

00:15:55.413 --> 00:15:59.613
So what they do is they, they
pack that into a design and they

00:15:59.613 --> 00:16:01.143
call it the promising zone design.

00:16:02.703 --> 00:16:07.143
Now, I'm gonna simplify it a little
bit, but they talk the, the general

00:16:07.143 --> 00:16:13.263
concept of this design is, okay, I'm
originally targeting 300 and I have 0.025

00:16:13.263 --> 00:16:14.073
at 300.

00:16:14.073 --> 00:16:19.803
Remember, that gave us 80% power for an
effect size of four that we're targeting.

00:16:21.033 --> 00:16:24.963
Now, in order to use this
result, I have to go backwards

00:16:24.963 --> 00:16:27.363
in time to an earlier time.

00:16:27.363 --> 00:16:29.913
I can't use this at 300.

00:16:31.233 --> 00:16:35.583
Because at 300, my conditional
power at 300 is either one or zero.

00:16:37.143 --> 00:16:44.733
So I have to do my look for adjusting
sample size earlier, like 200 so that

00:16:44.733 --> 00:16:47.583
I can use this neat mathematical trick.

00:16:47.583 --> 00:16:49.893
So I go back and I do an interim at two.

00:16:51.288 --> 00:16:55.488
Yes, you can do a superiority
analysis at that point, say, and

00:16:55.488 --> 00:16:58.008
they do talk about spending 0.001

00:16:58.008 --> 00:16:59.778
alpha for superiority there.

00:17:00.048 --> 00:17:02.148
That's your choice to do that.

00:17:02.148 --> 00:17:10.188
But at 200, I look to see if my
conditional power is above 0.5

00:17:10.188 --> 00:17:13.548
and let's, let's say we can
calculate exactly the threshold

00:17:13.548 --> 00:17:14.808
in the integral and it's 0.4.

00:17:16.578 --> 00:17:17.898
So 0.4

00:17:17.898 --> 00:17:24.528
or above, I can increase my sample size
from 300 to anything I want, uh, to three

00:17:24.528 --> 00:17:28.248
50 to 400, 500 without adjusting alpha.

00:17:29.718 --> 00:17:32.358
So let's say we can adjust it to 500.

00:17:32.358 --> 00:17:36.738
They talk about the case that the data
at that time point triggers exactly what

00:17:36.738 --> 00:17:39.198
your sample size is between 300 and 500.

00:17:39.408 --> 00:17:42.648
Turns out it very rarely picks
a sample size between there.

00:17:42.648 --> 00:17:44.388
It almost always goes to the biggest.

00:17:44.388 --> 00:17:45.408
You'll allow it to do.

00:17:45.808 --> 00:17:47.638
It also can be unblinding.

00:17:47.638 --> 00:17:51.598
If the sample size is three 40, you
know exactly what the effect size

00:17:51.598 --> 00:17:53.188
was 'cause of the conditional power.

00:17:53.338 --> 00:17:56.818
So let's simplify it in
this case to be that at.

00:17:56.848 --> 00:17:59.188
If the conditional power is above 0.4,

00:17:59.188 --> 00:18:07.858
we can increase the sample size to 500,
and our analysis at 500 uses 0.025.

00:18:09.238 --> 00:18:11.428
So we do this spend free.

00:18:12.508 --> 00:18:13.743
The crux of it is.

00:18:14.538 --> 00:18:21.138
The mathematical trick tells me when I can
increase my sample size, and it's this 0.4

00:18:21.138 --> 00:18:25.098
or above, and the design usually
does something like from 0.4

00:18:25.308 --> 00:18:26.358
to 0.9.

00:18:26.358 --> 00:18:29.748
In conditional power, I'm going
to increase the sample size

00:18:29.748 --> 00:18:33.798
to 500, but if it's above 0.9,

00:18:33.888 --> 00:18:36.858
I'm really liking my
chances of winning at 300.

00:18:36.858 --> 00:18:38.118
I'm gonna go with 300.

00:18:38.298 --> 00:18:41.328
I'm not going to increase it
because my conditional power's high.

00:18:42.303 --> 00:18:46.563
This neat mathematical trick allows
me to do this without any alpha spend.

00:18:46.563 --> 00:18:50.433
So the, the promising zone
design is do an interim at 200.

00:18:51.243 --> 00:18:55.743
If my p value at that
time's significant at 0.001,

00:18:56.073 --> 00:18:57.183
I call significant.

00:18:57.183 --> 00:18:58.023
My trial's over.

00:18:59.133 --> 00:19:02.073
If my conditional power is above 0.9,

00:19:02.073 --> 00:19:04.538
but I haven't won yet,
I'm gonna stick with 300.

00:19:06.048 --> 00:19:08.898
If my conditional powers between 0.4

00:19:08.898 --> 00:19:10.038
and 0.9,

00:19:10.038 --> 00:19:14.148
I'm gonna increase from 300 to 500,
giving me that additional power.

00:19:15.348 --> 00:19:19.188
Now, if my conditional
powers less than 0.4,

00:19:19.188 --> 00:19:25.218
I can't increase that was this
other place where my data are bad.

00:19:26.118 --> 00:19:31.488
Bad is, is a, is a bad term, uh,
for that, where it just largely

00:19:31.488 --> 00:19:35.598
means my conditional powers in this
region where if I want to increase

00:19:35.598 --> 00:19:37.668
sample size, I need to adjust alpha.

00:19:37.728 --> 00:19:42.558
And that's sort of, uh, we're not doing
that in the promising zone design.

00:19:42.558 --> 00:19:45.588
There's this, I'm gonna say I
think it's a bad approach, but

00:19:45.768 --> 00:19:46.638
we don't want to spend alpha.

00:19:47.733 --> 00:19:50.493
Which I think is not good design.

00:19:50.763 --> 00:19:54.693
Uh, your, your role is not to preserve
your alpha, your role is to create

00:19:54.693 --> 00:19:58.803
a good design, but that, that's
the, the premise of this design, and

00:19:58.803 --> 00:20:02.733
it's why it's attractive to people,
is you don't actually spend alpha.

00:20:02.913 --> 00:20:04.413
So if it's 0.4

00:20:04.413 --> 00:20:07.983
or lower, I have to stick with my 300.

00:20:08.928 --> 00:20:14.358
Knowing that my chances of success
are reasonably small, and boy, if my

00:20:14.358 --> 00:20:16.758
power, if my conditional power was 0.3,

00:20:16.758 --> 00:20:19.488
that's the case where I would
love to increase to 500.

00:20:19.908 --> 00:20:23.808
But the design, the mathematical
trick doesn't allow me to do it.

00:20:24.858 --> 00:20:28.908
This mathematical trick is making
my drug development decisions

00:20:28.968 --> 00:20:34.278
and me personally, that makes me
uncomfortable, um, in that scenario.

00:20:34.308 --> 00:20:36.258
But how does this work?

00:20:36.753 --> 00:20:37.983
How does this design work?

00:20:37.983 --> 00:20:40.143
So that's the design that I described.

00:20:40.713 --> 00:20:44.763
Uh, you could embed futility in there
and I'll, I'll leave that aside for now.

00:20:45.003 --> 00:20:48.393
Uh, I think that's a very good
idea with, within this scenario.

00:20:49.803 --> 00:20:56.163
So in this design, when you employ
that design, let's think about

00:20:56.163 --> 00:20:58.653
what happens in the trial at 200.

00:20:58.653 --> 00:21:02.193
You do this interim largely.

00:21:02.613 --> 00:21:07.473
If your effect size is
sort of north of about 5.5,

00:21:07.473 --> 00:21:14.583
remember if, if, if it's um, uh,
four has a power of 80% at that

00:21:14.583 --> 00:21:19.743
300, but if I'm north of about 5.5,

00:21:19.803 --> 00:21:22.418
I'm gonna win with the
group sequential 0.001.

00:21:23.298 --> 00:21:25.428
Oh, by the way, I said 0.025

00:21:25.428 --> 00:21:28.428
alpha because of that 0.001,

00:21:28.428 --> 00:21:29.988
it's 0.0249,

00:21:30.438 --> 00:21:33.618
um, uh, in that case,
but largely it's 0.025

00:21:33.618 --> 00:21:34.188
in that setting.

00:21:35.178 --> 00:21:39.078
So if I get to 200 and my
effect size is above 5.5,

00:21:39.078 --> 00:21:41.478
largely, I, I've already
won the trial and it's over.

00:21:43.248 --> 00:21:47.208
Now, if it's in that
rage region of about 5.5,

00:21:47.208 --> 00:21:49.038
down to about four.

00:21:50.718 --> 00:21:57.198
In that case, I'm trending with
90% conditional power and above.

00:21:57.348 --> 00:22:01.068
I'm gonna stick with 300 in that case.

00:22:01.608 --> 00:22:04.728
So my conditional power of, of 0.9

00:22:04.728 --> 00:22:09.198
is about effect size of four, uh, in that
case, which as originally powering for.

00:22:09.198 --> 00:22:13.248
So if I, if I end up north of that, I'm
really trending to a successful trial.

00:22:13.248 --> 00:22:18.588
Remember 200 patients into my
trial, I have a mean in the bank.

00:22:19.218 --> 00:22:24.018
A four or above gives me a conditional
power with only a hundred patients to

00:22:24.018 --> 00:22:26.388
go that I'm going to win the trial.

00:22:26.418 --> 00:22:29.328
And, and remember, I'm
powering at five, at 300.

00:22:29.598 --> 00:22:35.358
If I see something north of three,
I'm gonna win the trial at at 300.

00:22:37.338 --> 00:22:37.848
Okay?

00:22:38.298 --> 00:22:45.378
Now the promising zone is labeled
as that region where I'm between 0.4

00:22:45.378 --> 00:22:46.548
and 0.9

00:22:47.328 --> 00:22:48.228
in conditional power.

00:22:50.088 --> 00:22:55.068
When it falls in that case,
those are effect sizes.

00:22:55.278 --> 00:23:01.668
Just below this, I need three to win
sort of thing, and it's, it's about 2.7

00:23:02.238 --> 00:23:10.458
or above where that, where I'm gonna
have 40% chance of winning at 300, 2.75,

00:23:10.458 --> 00:23:10.998
let's call it.

00:23:10.998 --> 00:23:12.588
So 2.75.

00:23:12.588 --> 00:23:15.888
Up to four is the defined promising zone.

00:23:17.283 --> 00:23:21.663
That's the case where I'm free to
increase to 500, so I'm gonna go to 500.

00:23:21.663 --> 00:23:25.413
In that case, if my effect is 2.7

00:23:25.413 --> 00:23:31.233
or less, I have to stay at 300, so
I'm gonna go to 300 in the trial.

00:23:31.563 --> 00:23:33.123
That's what the design does.

00:23:33.123 --> 00:23:35.493
That's what it looks like
in the effect size of this.

00:23:36.348 --> 00:23:38.808
Now, what are the ramifications of this?

00:23:39.258 --> 00:23:41.838
First of all, I want you to
think about trials that go

00:23:41.838 --> 00:23:43.848
forward into the promising zone.

00:23:43.998 --> 00:23:48.408
Have conditional power at
300 to win between, uh, 0.4

00:23:48.408 --> 00:23:49.068
and 0.9.

00:23:49.818 --> 00:23:52.188
Most of those trials would've won at 300.

00:23:54.138 --> 00:23:56.893
More than 50% would've won at 300.

00:23:57.313 --> 00:24:00.978
So when you go to the promising zone and
you go to 500, it's gonna increase that.

00:24:02.328 --> 00:24:05.088
So you simulate exactly that design.

00:24:05.118 --> 00:24:10.758
Remember our old case of four
that had power of about 82%?

00:24:11.508 --> 00:24:13.818
That goes up about 4%.

00:24:16.008 --> 00:24:19.788
In the case of four, we
increase our power by about 4%.

00:24:21.438 --> 00:24:24.378
We also, I, I, I'll sort of stop there.

00:24:24.648 --> 00:24:26.208
Our sample size.

00:24:26.643 --> 00:24:32.493
Our average sample size increases from
300 in the fixed sample size to 342.

00:24:35.193 --> 00:24:41.523
That's, that's not a sort of small
increase, but if you ran a trial

00:24:41.523 --> 00:24:47.973
of 342 patients, you would get the
same four ish percent increase,

00:24:47.973 --> 00:24:49.443
four to five increase in power.

00:24:51.993 --> 00:24:52.983
Is that efficient?

00:24:52.983 --> 00:24:54.273
No, it's not very efficient.

00:24:55.158 --> 00:25:00.168
Yes, it's better than the fixed design
and power, but not really in efficiency.

00:25:00.858 --> 00:25:03.468
It's a, it's a marginal benefit.

00:25:03.468 --> 00:25:08.388
In the case of four, now you're
80% powered, and now you're,

00:25:08.388 --> 00:25:10.188
you're increasing that four or 5%.

00:25:10.188 --> 00:25:11.988
So I'm not gonna say that's bad in that.

00:25:12.648 --> 00:25:14.478
What about that case of three?

00:25:15.258 --> 00:25:20.028
Three was the case that we were
lingering just below 60% power and

00:25:20.028 --> 00:25:25.668
500 was gonna give us, in that case,
it was gonna give us 80% power.

00:25:25.668 --> 00:25:33.288
If we ran 500, the power in that case
increases about 6%, and our sample

00:25:33.288 --> 00:25:37.068
size increases by 55 on average, 355.

00:25:38.838 --> 00:25:45.378
We also in, in increase about, there's
about a 2% chance that you lose the trial

00:25:45.378 --> 00:25:47.808
at 500 that you would've won of 300.

00:25:48.978 --> 00:25:49.773
That that can happen.

00:25:50.493 --> 00:25:52.863
If you just would've gone
to 300, you would've won.

00:25:52.863 --> 00:25:56.943
But because you went to the end, it
actually ended up, uh, getting worse.

00:25:57.003 --> 00:26:02.703
So you get this sort of plus 5%, but
you lose kind of 2%, I'm sorry, plus 6%,

00:26:02.703 --> 00:26:07.443
but you lose 2% that you would've wanted
300, and you spend 55 patients for that.

00:26:08.583 --> 00:26:13.503
Again, 355 patient trial has more
power than the promising zone design.

00:26:14.793 --> 00:26:18.783
On average, if you as a company
run this promising zone over and

00:26:18.783 --> 00:26:22.683
over, you're, you're not efficient
with the use of your patients.

00:26:25.083 --> 00:26:25.833
Okay.

00:26:26.163 --> 00:26:32.373
So, um, and this is a, this is sort of a
perfect case of, of the promising zone.

00:26:32.703 --> 00:26:34.653
Is it better than a fixed trial?

00:26:34.653 --> 00:26:35.733
I, I don't know.

00:26:35.733 --> 00:26:41.553
I'm not so sure, uh, in that
case, but let's, uh, the, the

00:26:41.553 --> 00:26:43.213
attraction to it is sort of this.

00:26:44.283 --> 00:26:48.123
Bizarre allergic reaction
to spending alpha.

00:26:48.543 --> 00:26:51.663
Again, the goal of the trial
is not to preserve your alpha.

00:26:51.813 --> 00:26:55.923
So many companies come to us and say, oh,
we we're doing a promising zone design.

00:26:56.823 --> 00:26:58.743
Oh, because we don't wanna spend alpha.

00:26:59.673 --> 00:27:03.393
Well, it doesn't mean it's a good
design, it doesn't mean it's efficient.

00:27:03.633 --> 00:27:05.703
Have you compared it to other designs?

00:27:05.733 --> 00:27:10.053
Let's take exactly that trial and
say, under those parameters, let's

00:27:10.053 --> 00:27:11.373
do a group sequential design.

00:27:11.823 --> 00:27:13.623
Let's spend alpha.

00:27:13.983 --> 00:27:19.263
Now, I should stop saying spend, because
spend sort of means like it's gone.

00:27:19.473 --> 00:27:23.613
You allocate it across
this, you still get 2.5%

00:27:23.613 --> 00:27:24.063
alpha.

00:27:24.303 --> 00:27:25.893
You haven't lost any of it.

00:27:25.953 --> 00:27:28.713
It's all there in the design.

00:27:28.893 --> 00:27:31.263
You just use it at different
points in the design.

00:27:31.263 --> 00:27:32.613
Hopefully in an efficient way.

00:27:33.513 --> 00:27:39.693
We're gonna use a fairly non-controversial
Kim Deme two spending function.

00:27:40.848 --> 00:27:45.048
And we'll use exactly the 200
case that the promising Zone

00:27:45.048 --> 00:27:47.688
did, and then we'll do 300.

00:27:47.688 --> 00:27:50.058
That was original sample size four.

00:27:50.058 --> 00:27:52.698
We'll allow 400 and 500.

00:27:53.388 --> 00:27:57.258
So here's a design that has the exact
same range, the same minimum, the

00:27:57.258 --> 00:28:01.548
same maximum, and we'll use a Kim
Deme spending function over those.

00:28:01.788 --> 00:28:06.078
By the way, the final
alpha at 500 is 0.0173.

00:28:07.008 --> 00:28:11.208
And again, that gets people so
nervous that that's less than 0.025

00:28:11.928 --> 00:28:13.938
and that's what they call the spend.

00:28:14.238 --> 00:28:14.658
Yeah.

00:28:14.838 --> 00:28:16.068
But it's all used.

00:28:16.458 --> 00:28:22.128
It's you, you use all of that Alpha
200, 300, 400, you have, I, it's, it's

00:28:22.128 --> 00:28:28.338
so strange to me that, that people
have this allergic to reaction to using

00:28:28.338 --> 00:28:30.648
their alpha in, in the efficient ways.

00:28:30.978 --> 00:28:32.838
So what does that design do?

00:28:33.198 --> 00:28:37.458
Compared to the promising zone design,
fairly simple, straightforward, right?

00:28:37.458 --> 00:28:43.518
I haven't even baked in
futility at 400, uh, sorry.

00:28:43.848 --> 00:28:49.668
At the effect of four, remember, we're 80,
a little over 80% power for 300 patients.

00:28:50.868 --> 00:28:56.178
The promising zone increased that,
let's call it 5% for an average

00:28:56.178 --> 00:28:57.768
of almost 50 patients, more.

00:28:58.833 --> 00:29:04.323
The group sequential design
at four has a power of 95%.

00:29:05.763 --> 00:29:14.163
It goes from about 82 to 95% more than
double the increase in power at three.

00:29:14.398 --> 00:29:18.663
At at at the effect size
original goal of four.

00:29:19.953 --> 00:29:22.953
The average sample size is 310.

00:29:24.768 --> 00:29:28.758
The promising zone, average
sample size in that case was 3 42.

00:29:30.948 --> 00:29:33.798
No different than a fixed
sample size of 3 42.

00:29:34.638 --> 00:29:40.128
The av, the, the power of 310 patients
at that case would be about 83%.

00:29:41.268 --> 00:29:42.528
Ours is 95.

00:29:44.118 --> 00:29:46.698
That's an efficient use
of your sample size.

00:29:46.878 --> 00:29:50.178
If you're running five of these
trials, 10 of these trials.

00:29:50.538 --> 00:29:54.168
Uh, if the industry's running many
of these trials, it's an incredibly

00:29:54.168 --> 00:30:03.618
efficient use of, of the, the sample size,
patience, time, cost in that it's much

00:30:03.618 --> 00:30:09.288
better than the promising zone design,
and it allocates alpha across these.

00:30:10.878 --> 00:30:15.348
Remember that, that that little three,
the effect of three, that boy, we'd

00:30:15.348 --> 00:30:17.628
love to go to 500 In that circumstance.

00:30:18.228 --> 00:30:23.478
The power increase in that case is
more than 20% increase in power.

00:30:23.628 --> 00:30:26.358
It almost gets to that 80% that we get.

00:30:26.358 --> 00:30:28.098
It's, it's about 78%.

00:30:29.418 --> 00:30:33.828
It's almost a 20% increase in power
that the promising zone increase 5%.

00:30:34.023 --> 00:30:37.953
It and the promising zone
increased it by 55 patients.

00:30:37.953 --> 00:30:43.053
This increases it to 3 83, but
you get a huge increase in power.

00:30:43.353 --> 00:30:47.883
You almost get the power of a 500
patient trial for an increase of 83.

00:30:47.883 --> 00:30:50.613
Again, incredibly efficient use of alpha.

00:30:52.983 --> 00:30:56.973
Same thing happens for an effect size
even of two in that circumstance.

00:30:57.018 --> 00:31:00.468
Of course, the other beautiful thing
is your, your effect size is five.

00:31:00.468 --> 00:31:03.198
Your power is almost a hundred
percent, and your average sample

00:31:03.198 --> 00:31:09.648
size is two 50 because you win at
the 200 patient look 60% of the time.

00:31:11.058 --> 00:31:12.948
Again, it's incredibly efficient.

00:31:12.948 --> 00:31:18.858
Use the group Sequential design
dominates the promising zone design.

00:31:20.513 --> 00:31:24.048
I, I don't care if you spent
Alpha it, it just doesn't matter.

00:31:24.228 --> 00:31:27.858
When you simulate and compare that DI
design to the promising zone, there's

00:31:27.858 --> 00:31:29.568
no way you'd pick the promising zone.

00:31:31.428 --> 00:31:34.878
Okay, now, um.

00:31:36.273 --> 00:31:40.503
Embedding within that, people get
nervous Well, uh, you might go to 500.

00:31:40.773 --> 00:31:43.023
You need to embed futility within that.

00:31:43.713 --> 00:31:47.433
Add futility rules at
the 200, 300, 400, 500.

00:31:47.433 --> 00:31:51.693
Simulate it, optimize
it in those scenarios.

00:31:52.143 --> 00:31:54.183
Create a much more efficient design.

00:31:54.213 --> 00:31:58.233
The amazing thing about the group
sequential design, and, and by the

00:31:58.233 --> 00:32:01.383
way, I'm a card carrying Bayesian.

00:32:01.818 --> 00:32:06.138
Uh, but I live in the world of type
1 error in phase III and I, I did

00:32:06.138 --> 00:32:08.148
that discussion with Frank Harrell.

00:32:08.358 --> 00:32:12.888
He gets mad at us for doing this,
but it's the world we live in and

00:32:12.888 --> 00:32:14.388
we do group sequential designs.

00:32:14.388 --> 00:32:18.498
We explore designs and look at the OPAL
So I'm not somebody standing up here

00:32:18.498 --> 00:32:23.178
saying frequentist is great and all that,
but the design is pretty darn efficient

00:32:23.508 --> 00:32:28.098
in the world where you have to control
type 1 error It's really darn efficient.

00:32:28.443 --> 00:32:35.523
In that, those circumstances, uh, within
that, so the, the group sequential design

00:32:35.523 --> 00:32:41.583
can be incredibly efficient use of it,
tied with futility, tied with simulation.

00:32:41.583 --> 00:32:48.003
Compare it, uh, in those cases, we,
we get so many clients coming in

00:32:48.003 --> 00:32:52.863
that, that don't wanna spend the alpha
and they write in a promising zone.

00:32:52.863 --> 00:32:55.143
And it seems like this amazing thing.

00:32:55.143 --> 00:32:55.803
Oh, sorry.

00:32:55.923 --> 00:32:57.153
I what I was gonna say.

00:32:57.813 --> 00:33:03.273
When you do this group sequential
design at 200, where again you win

00:33:03.273 --> 00:33:08.583
north of five, you stop for futility
based on some point where you

00:33:08.583 --> 00:33:11.493
decide as the sponsor where that is.

00:33:11.493 --> 00:33:14.343
You don't use the mathematical
trick to say it's 0.4.

00:33:15.873 --> 00:33:19.563
You decide at what values
you would like to keep going.

00:33:20.148 --> 00:33:25.368
In those circumstances, it
creates a promising zone between

00:33:25.368 --> 00:33:27.138
your futility and we won.

00:33:27.318 --> 00:33:28.968
That's the region we'd like to keep going.

00:33:28.968 --> 00:33:32.418
And then at 300, it creates another
one, another promising zone.

00:33:33.798 --> 00:33:39.618
The difference in the designs is you
make the decision of whether 300 is the

00:33:39.618 --> 00:33:42.858
right sample size when you get to 300.

00:33:44.223 --> 00:33:48.273
You don't have this weird mathematical
trick say, you know, we need to go

00:33:48.273 --> 00:33:54.753
back to 200 to decide if 300 is the
right sample size, because there's

00:33:54.753 --> 00:34:00.933
this cool mathematical trick about good
and bad data and spending alpha, the

00:34:00.933 --> 00:34:04.798
promising zone's been out there and
it it, it's a siren song for people.

00:34:05.018 --> 00:34:05.558
Oh my God.

00:34:05.558 --> 00:34:08.973
There's this adaptive sample size
approach that doesn't spend alpha.

00:34:09.483 --> 00:34:10.863
It's a siren song for people.

00:34:12.558 --> 00:34:17.838
Simulate the design, compare it to a
fixed design, compare it to a group

00:34:17.838 --> 00:34:20.208
sequential design that allocates Alpha.

00:34:22.338 --> 00:34:30.198
We at Berry have never once had a
scenario where a sponsor comes in

00:34:31.158 --> 00:34:35.208
where the promising zone design is
better than a group sequential design.

00:34:36.078 --> 00:34:36.588
Zero.

00:34:37.038 --> 00:34:39.648
If you have one out there, show it to me.

00:34:40.098 --> 00:34:42.168
Uh, I, I, I, I'd love to see it.

00:34:42.233 --> 00:34:44.688
I'd, I'd love to see
this rare bird out there.

00:34:46.098 --> 00:34:51.348
Now, interestingly, of course, in the
whole thing, and it, it's more than I can

00:34:51.348 --> 00:34:56.988
do in this, this, uh, this episode, but
I, that was a circumstance where you had

00:34:56.988 --> 00:34:58.848
complete information on every patient.

00:34:58.848 --> 00:34:58.908
It.

00:35:00.138 --> 00:35:02.448
Now, most trials don't have that.

00:35:02.598 --> 00:35:04.398
Our endpoint is 12 months.

00:35:04.398 --> 00:35:06.078
Our endpoint is six months.

00:35:06.498 --> 00:35:09.348
Our endpoint is 90 day modified ranking.

00:35:09.648 --> 00:35:13.608
Uh, a number of scenarios that when
we do our interim, we have a number of

00:35:13.608 --> 00:35:15.768
patients with incomplete information.

00:35:18.108 --> 00:35:22.488
You can do group sequential designs
with incomplete information now.

00:35:23.598 --> 00:35:25.668
We call those Goldilocks designs.

00:35:25.668 --> 00:35:31.338
And you can look up, um, uh, Broley
O'Connor and Barry Paper on, uh,

00:35:31.368 --> 00:35:35.778
Goldilocks design, where largely you're
employing group sequential techniques.

00:35:35.778 --> 00:35:38.958
When you get to 300, you make
the decision is 300, right?

00:35:38.958 --> 00:35:40.878
You don't know the final answer yet.

00:35:42.108 --> 00:35:43.638
You don't know your p value.

00:35:43.908 --> 00:35:48.168
You can use conditional power, predictive
power to say, my probability's high.

00:35:48.168 --> 00:35:49.158
I'm gonna win right here.

00:35:50.448 --> 00:35:51.918
And then I stop enrolling.

00:35:51.918 --> 00:35:57.978
I get complete data, and I do my P-value
calculation With incomplete data.

00:35:58.038 --> 00:36:01.458
That's still better than the promising
zone because remember the promising

00:36:01.458 --> 00:36:06.828
zone goes back to 200 and says, I
now need to forecast 300, but it has

00:36:06.828 --> 00:36:12.288
incomplete information because at 200 it
is also dealing with the same problem.

00:36:13.593 --> 00:36:18.363
In all of those same circumstances, the
promising zone has never been better than

00:36:18.363 --> 00:36:21.513
a Goldilocks group sequential type design.

00:36:24.273 --> 00:36:30.333
Now, it, it's, it's, it's one
of these circumstances that math

00:36:30.333 --> 00:36:33.813
triggers the design, and that just
makes me incredibly uncomfortable.

00:36:35.388 --> 00:36:38.478
This neat mathematical result
that you don't inflate.

00:36:38.478 --> 00:36:40.278
Type one error at 0.37

00:36:40.278 --> 00:36:40.968
or 0.4

00:36:40.968 --> 00:36:41.988
or 0.43

00:36:42.258 --> 00:36:44.358
should not be your drug
development decision.

00:36:44.718 --> 00:36:49.008
Don't force a decision earlier to do
this neat little mathematical trick.

00:36:52.878 --> 00:36:57.408
Don't make your decision about
where you're gonna go around

00:36:57.408 --> 00:37:01.153
Minneapolis two hours earlier.

00:37:03.888 --> 00:37:08.418
Now it's not all that different
of a scenario because we

00:37:08.418 --> 00:37:09.888
have incomplete information.

00:37:10.368 --> 00:37:13.278
So when I get to the top
part of Minneapolis, I have

00:37:13.278 --> 00:37:15.768
to decide 4 94 to the west.

00:37:15.948 --> 00:37:18.108
I can go 94 through the city.

00:37:18.348 --> 00:37:20.658
I can go a little bit east of that.

00:37:21.078 --> 00:37:21.648
Um.

00:37:22.203 --> 00:37:27.213
Now I have incomplete information when
I get there, so right about the, the

00:37:27.213 --> 00:37:29.013
point I have to make this decision.

00:37:29.373 --> 00:37:34.353
The app is gonna tell me the relative time
for each of these, and ru and, and some

00:37:34.353 --> 00:37:36.333
rush hours, rush hour time and everything.

00:37:36.333 --> 00:37:40.113
Think you're pretty bad in Minneapolis
in that, so I'm still dealing with

00:37:40.113 --> 00:37:44.463
incomplete information, but I'd rather
do it close to the time I have to

00:37:44.463 --> 00:37:48.183
make the decision rather than when
I leave my house two hours earlier.

00:37:49.458 --> 00:37:52.518
It just, it, it, it can't be
possible that I can make better

00:37:52.518 --> 00:37:54.018
decision two hours earlier.

00:37:54.498 --> 00:37:57.408
And in this adaptive sample
size thing, that's what is done

00:37:57.408 --> 00:37:58.878
in the promising zone design.

00:38:00.648 --> 00:38:07.008
Sure, it's better than a fixed sample
size, but once you, once you get

00:38:07.008 --> 00:38:14.538
into this world and say, I am in a
scenario where I could go to 500, if

00:38:14.538 --> 00:38:16.698
the promising zone says Go to 500.

00:38:17.373 --> 00:38:20.043
I could have a flexible sample size.

00:38:20.673 --> 00:38:23.193
Do it right, simulate it.

00:38:23.193 --> 00:38:25.233
Explore the different approaches.

00:38:25.533 --> 00:38:27.633
Don't just write in the promising zone.

00:38:27.663 --> 00:38:33.843
It's, you know, in the siren song, they
run into the rocks and it's not good.

00:38:34.563 --> 00:38:35.193
Compare it.

00:38:35.193 --> 00:38:39.303
If you find a case where the promising
zone does better, let me know.

00:38:39.513 --> 00:38:40.743
That's really cool.

00:38:41.043 --> 00:38:41.553
But.

00:38:41.883 --> 00:38:43.893
We have yet to find that one.

00:38:44.403 --> 00:38:46.443
Make the decision when you need to.

00:38:46.653 --> 00:38:50.943
Don't make it earlier than that,
and be efficient with your use

00:38:50.943 --> 00:38:53.583
of alpha here in the interim.

00:38:54.183 --> 00:38:58.143
We take that seriously and we
try to use our alpha efficiently.

00:38:58.473 --> 00:39:01.413
We also try to drive and get
there in the shortest amount

00:39:01.413 --> 00:39:03.393
of time, and I'm heading there.

00:39:04.278 --> 00:39:09.948
In a couple weeks, we'll be making that
trip, um, in there and I plan to have

00:39:09.948 --> 00:39:14.148
my app open, telling me, me, which way
to go and not making those decisions

00:39:14.148 --> 00:39:16.098
before I get to that point in the road.

00:39:17.208 --> 00:39:20.718
So I thank you all again for
joining me here in the interim

00:39:21.348 --> 00:39:25.128
and till our next interim, cheers.