WEBVTT

NOTE
This file was generated by Descript 

00:00:00.000 --> 00:00:03.410
Judith: Welcome to Berry's In the
Interim podcast, where we explore the

00:00:03.410 --> 00:00:07.320
cutting edge of innovative clinical
trial design for the pharmaceutical and

00:00:07.320 --> 00:00:09.500
medical industries, and so much more.

00:00:10.110 --> 00:00:10.760
Let's dive in.

00:00:11.825 --> 00:00:17.585
Scott Berry: Ah, welcome everybody
to, in the Interim, a podcast on all

00:00:17.585 --> 00:00:22.235
things science of clinical trials,
statistics of clinical trials.

00:00:22.235 --> 00:00:22.295
I.

00:00:22.850 --> 00:00:28.670
And, uh, we, we, we pose this question
of, in the interim, in adaptive

00:00:28.670 --> 00:00:32.660
trials, and so we've got some talk
of other adaptive trials today.

00:00:33.110 --> 00:00:35.060
So I have some really cool guests.

00:00:35.060 --> 00:00:35.420
Today.

00:00:35.420 --> 00:00:38.360
We're gonna talk about a
particular paper that came out.

00:00:38.420 --> 00:00:44.810
We'll introduce the paper and I
have the, my co-authors here, so Dr.

00:00:44.810 --> 00:00:45.380
Kurt Veley.

00:00:46.055 --> 00:00:50.765
Who, uh, sometimes hosts this show
and has done a number of the podcasts.

00:00:51.125 --> 00:00:52.025
And Dr.

00:00:52.025 --> 00:00:53.435
Joe Marion and Dr.

00:00:53.435 --> 00:00:55.325
Lindsey Berry have joined me today.

00:00:55.325 --> 00:00:56.855
So welcome everybody.

00:00:58.460 --> 00:00:58.970
Joe Marion: Hey Scott.

00:00:59.795 --> 00:01:00.305
Lindsay Berry: Thanks.

00:01:01.145 --> 00:01:03.995
Scott Berry: So what is the, the paper?

00:01:04.025 --> 00:01:06.305
The paper is, it's been published.

00:01:06.365 --> 00:01:13.625
The title is Optimal Sample Size
Division in Two Stage Seamless Designs.

00:01:15.365 --> 00:01:21.365
So as, as Joe just reminded me,
the math isn't necessarily new

00:01:21.365 --> 00:01:24.275
here, but I somewhat to me a bit.

00:01:24.815 --> 00:01:26.195
Uh, inaccessible.

00:01:26.645 --> 00:01:32.705
And so what is the, what is the problem
and, and what are we doing historically?

00:01:32.915 --> 00:01:37.115
Louis Scheer referred to this
as there's learning trials

00:01:37.115 --> 00:01:38.645
and there's confirming trials.

00:01:39.155 --> 00:01:41.225
You run a phase two trial.

00:01:41.245 --> 00:01:46.735
You look at the data, you make decisions,
you run a confirmatory two arm phase

00:01:46.735 --> 00:01:50.845
three trials, and these don't mix well.

00:01:50.845 --> 00:01:53.005
The world is changing a little bit.

00:01:53.095 --> 00:01:58.165
And, and as, uh, Don Berry always
says to us, you're always learning

00:01:58.315 --> 00:02:00.745
and you're always confirming,
you're always doing both.

00:02:00.745 --> 00:02:06.235
So we're gonna meld these together into
a seamless phase two three trial design.

00:02:07.130 --> 00:02:12.650
So a very simple example of this
might be three doses in a placebo.

00:02:13.010 --> 00:02:19.010
In stage one of the trial, we're gonna
confuse and say phase two, stage one.

00:02:19.010 --> 00:02:24.290
So stage one of the trial, maybe
50 patients on each arm, fixed

00:02:24.290 --> 00:02:27.530
randomization, pick a dose.

00:02:27.845 --> 00:02:29.165
Seamlessly.

00:02:29.435 --> 00:02:35.345
The next day, you're enrolling to that
dose in placebo, maybe a hundred patients.

00:02:35.345 --> 00:02:43.745
On each of those, the, the selected
dose and the, uh, placebo seamless

00:02:43.745 --> 00:02:48.485
refers to this moving from stage
one to stage two of the trial.

00:02:48.755 --> 00:02:54.065
This is made by algorithm or by DSMB,
but made within a couple days, and you

00:02:54.065 --> 00:02:55.745
now you're just enrolling part two.

00:02:56.825 --> 00:03:03.395
And these will be inferentially seamless
in that the analysis at the end of stage

00:03:03.395 --> 00:03:08.855
two is going to include the patients
from the first stage of the trial

00:03:09.365 --> 00:03:11.615
and in the second stage of the trial.

00:03:12.905 --> 00:03:15.485
So these trials, you'll,
you'll read about them.

00:03:15.485 --> 00:03:16.925
You'll see about them.

00:03:17.285 --> 00:03:19.895
What was inaccessible to me is.

00:03:20.195 --> 00:03:27.335
I we're used to group sequential
type trials where you spend alpha.

00:03:28.025 --> 00:03:32.135
So let me, I, I'm, I'm speaking
way too much and I, I said I

00:03:32.135 --> 00:03:33.455
wasn't, I was gonna be the ho.

00:03:33.935 --> 00:03:40.595
So, Kurt, why do we have to adjust
alpha at the end when we're combining

00:03:40.595 --> 00:03:42.995
stage 1 and stage 2 together?

00:03:44.400 --> 00:03:48.305
Kert Viele: Uh, so I mean, if, if
you are looking at multiple doses,

00:03:48.515 --> 00:03:52.850
you have multiple chances for
each of those doses to be picked

00:03:52.850 --> 00:03:54.785
and go on into the next phase.

00:03:55.280 --> 00:04:01.760
When you're actually using that data, you
expect that whatever dose you pick, it

00:04:01.760 --> 00:04:07.100
might have a slight upward bias because
it was picked among multiple options.

00:04:07.520 --> 00:04:10.130
And so whenever you see that
kind of cherry picking, I.

00:04:10.520 --> 00:04:13.550
You expect that there needs
to be some kind of alpha

00:04:13.550 --> 00:04:14.990
adjustment that goes with it.

00:04:15.380 --> 00:04:20.030
And so what we tried to do here
was to have nice, simple formulas

00:04:20.180 --> 00:04:21.890
for here is the adjustment.

00:04:21.890 --> 00:04:26.060
Here's something, you know, you take two
seconds on your computer and boom, this

00:04:26.060 --> 00:04:30.320
is the threshold that you need and you'll
control alpha You'll control type 1 error.

00:04:32.380 --> 00:04:32.800
Scott Berry: Okay.

00:04:32.800 --> 00:04:38.020
When I, when I've seen these
historically, we've designed these trials.

00:04:38.350 --> 00:04:41.260
There's a procedure out there
for controlling type one error.

00:04:41.620 --> 00:04:44.140
Uh, the closed testing
procedure Bretts it all.

00:04:45.515 --> 00:04:45.635
I.

00:04:46.280 --> 00:04:46.730
I.

00:04:46.730 --> 00:04:53.000
find it awkward, somewhat inaccessible,
that you combine the P value from

00:04:53.000 --> 00:04:56.810
both parts of this, and we're gonna
come back to this, but it doesn't

00:04:56.810 --> 00:05:02.210
do what we do in a group sequential
trial where we, we understand you

00:05:02.210 --> 00:05:07.760
need to control type one error and
we could adjust alpha at the end.

00:05:07.910 --> 00:05:10.610
Can we figure out what alpha?

00:05:11.255 --> 00:05:14.495
We need at the end
that's lower than 0.025.

00:05:14.555 --> 00:05:18.005
We're gonna talk about one-sided
type one error in that.

00:05:18.005 --> 00:05:20.795
That's lower than that,
but, but the closed testing

00:05:20.795 --> 00:05:22.595
procedure doesn't give you that.

00:05:22.865 --> 00:05:24.725
So can you do this?

00:05:25.595 --> 00:05:30.665
So this, this was, then it becomes very
accessible, almost like group sequential.

00:05:30.845 --> 00:05:33.065
So Joe, how did we.

00:05:34.280 --> 00:05:36.410
Uh, you know, I, I know
this math is out there.

00:05:36.410 --> 00:05:38.180
How do you solve this problem?

00:05:38.180 --> 00:05:43.280
How do you calculate an alpha at the
end of stage two given the trial design?

00:05:43.975 --> 00:05:46.940
Joe Marion: I mean, do, maybe we
should tell the story a little bit

00:05:46.940 --> 00:05:48.920
of like how this came to be, right?

00:05:48.920 --> 00:05:50.480
Like you, you came to us.

00:05:50.825 --> 00:05:53.945
As like a company with this,
uh, with this question, right.

00:05:54.005 --> 00:05:58.325
Uh, we have a weekly meeting where we
talk about statistical topics, cool things

00:05:58.325 --> 00:06:02.015
we're doing in designs, things, hard
problems, we don't know how to solve.

00:06:02.015 --> 00:06:05.645
And, and you, you brought this
topic, you said, I want to figure

00:06:05.645 --> 00:06:06.725
out how to do this adjustment.

00:06:07.265 --> 00:06:10.055
And so I think, and this is
something that like Lindsay had

00:06:10.055 --> 00:06:11.795
worked on a long time ago, right?

00:06:11.795 --> 00:06:13.925
Lindsay, when, when did you
first start working on this?

00:06:14.780 --> 00:06:15.620
Lindsay Berry: Yeah, that's right.

00:06:15.620 --> 00:06:21.020
I worked on this a little bit in
my undergrad at, uh, UT Austin.

00:06:21.020 --> 00:06:25.760
I was working on my thesis with,
um, Peter Mueller at ut, and we.

00:06:26.600 --> 00:06:34.010
We addressed this and the approach we took
there was using simulations to find the

00:06:34.010 --> 00:06:38.360
adjusted alpha threshold that you would
use at the end of a phase three study.

00:06:38.900 --> 00:06:41.630
Um, which is one approach
to, to doing this.

00:06:41.630 --> 00:06:47.240
And, um, you know, at the time I wasn't
aware of the math, you know, it exists

00:06:47.345 --> 00:06:49.235
Joe Marion: speaks to that
point, right, that we did.

00:06:49.445 --> 00:06:51.005
We just like didn't know
this was out there, right?

00:06:51.275 --> 00:06:51.455
Yeah.

00:06:52.160 --> 00:06:52.580
Mm-hmm.

00:06:52.950 --> 00:06:54.420
Scott Berry: So what is the math?

00:06:54.660 --> 00:06:59.100
How, how do you, so simulation,
straightforward, but um,

00:06:59.190 --> 00:07:00.600
how do you do the math?

00:07:00.630 --> 00:07:01.080
What?

00:07:01.110 --> 00:07:02.250
Can you set this up?

00:07:03.050 --> 00:07:07.010
Joe Marion: Yeah, so, so this comes
from, uh, group sequential theory,

00:07:07.460 --> 00:07:10.760
and I think Kurt and I both, after
you, you gave that talk, I think Kurt

00:07:10.760 --> 00:07:13.670
and I both went away and tinkered
individually and within a week we

00:07:13.670 --> 00:07:15.380
both had a solution to this, right?

00:07:16.070 --> 00:07:20.480
And the idea is that at the interim
analysis, you have a z statistic, your

00:07:20.480 --> 00:07:22.730
data can be represented by a Z statistic.

00:07:23.360 --> 00:07:28.910
Um, that sort of, uh, is approximately
normal and you have a Z statistic in

00:07:29.270 --> 00:07:35.660
each of your treatment arms, and then
you can ident and because you know that's

00:07:35.660 --> 00:07:40.070
approximately normal, you can get a closed
expression for the CDF of the maximum.

00:07:40.520 --> 00:07:43.340
So the math assumes you choose the
best dose, the one with the largest

00:07:43.340 --> 00:07:47.840
Z value, and then you can use, and
then you know that your data that

00:07:47.840 --> 00:07:49.340
you collect in the second half.

00:07:49.895 --> 00:07:52.325
Also follow a z statistic.

00:07:52.325 --> 00:07:57.365
And so you can, uh, use Quadrature to
figure out sort of the exact correction

00:07:57.365 --> 00:08:01.715
you need, um, controlling for the
correlation between the, the data at

00:08:01.715 --> 00:08:04.895
the interim analysis, and then the
full data set that you have at the end.

00:08:05.435 --> 00:08:06.785
So that's, that's kind of the math.

00:08:06.785 --> 00:08:12.185
It's, it's nice, um, you know,
uh, normal theory kinds of stuff.

00:08:12.185 --> 00:08:16.115
I think it's three or four
lines, um, worth of math.

00:08:16.145 --> 00:08:17.615
Uh, it's very elegant.

00:08:17.615 --> 00:08:18.965
It produces a very nice solution.

00:08:19.790 --> 00:08:23.865
Kert Viele: I, I think one of the,
like Scott, if I, yeah, the um.

00:08:24.315 --> 00:08:28.065
One of the things I thought really
is nice about the intuition here is

00:08:28.065 --> 00:08:30.525
it depends on where the interim is.

00:08:30.795 --> 00:08:35.745
So if the interim is very, very
early, so you do a really tiny

00:08:35.745 --> 00:08:39.825
phase two and a big phase three,
you're not reusing a lot of data.

00:08:39.885 --> 00:08:42.855
You don't have to take a penalty
because there's virtually no.

00:08:43.145 --> 00:08:43.775
Reuse.

00:08:44.345 --> 00:08:47.975
On the other hand, if the interim
is very, very late, you basically

00:08:47.975 --> 00:08:50.165
spend all of your time in phase two.

00:08:50.525 --> 00:08:53.915
It's not quite a bon ferone,
but it gets pretty close to it.

00:08:54.185 --> 00:08:59.015
So the extremes, the real question is,
well, where am I gonna put the interim?

00:08:59.285 --> 00:09:02.135
And I think this gets into the part
where the paper starts to break

00:09:02.135 --> 00:09:06.995
a little new ground, is knowing
what to do with a fixed interim.

00:09:07.265 --> 00:09:09.785
Where do you put the interim
to get the optimal design?

00:09:10.800 --> 00:09:11.070
Scott Berry: Yep.

00:09:11.250 --> 00:09:11.520
Yep.

00:09:12.090 --> 00:09:14.640
So you brought that up, the two extremes.

00:09:14.640 --> 00:09:20.370
Suppose we ran a seamless two three
trial where all I, I gave an example of

00:09:20.370 --> 00:09:23.940
three doses, uh, 50 patients on each.

00:09:23.945 --> 00:09:27.840
But if you ran all three
doses to the end of stage two.

00:09:28.250 --> 00:09:31.610
And you go to the end and now
you can pick any one of the doses

00:09:31.610 --> 00:09:33.380
to be statistically significant.

00:09:33.860 --> 00:09:36.170
You get an alpha that comes out of this.

00:09:36.170 --> 00:09:37.670
It's essentially Bonferroni.

00:09:37.670 --> 00:09:39.230
You get 0.0,

00:09:39.590 --> 00:09:44.150
uh, 0 8, 3 3, so you know, 0.025

00:09:44.150 --> 00:09:44.840
divided by three.

00:09:44.870 --> 00:09:46.400
'cause you carry them
all the way to the end.

00:09:47.450 --> 00:09:51.950
If you picked a dose and you didn't enroll
any patients in stage one and you just,

00:09:51.950 --> 00:09:54.145
you run a two arm trial, you get 0.025.

00:09:55.550 --> 00:10:03.080
So this calculates as a fraction of the
stage one size, what your alpha spend is.

00:10:03.320 --> 00:10:05.030
And we have R code for this.

00:10:05.030 --> 00:10:08.450
By the way, did we, did
we provide the R code and,

00:10:08.450 --> 00:10:09.320
and so people can

00:10:09.320 --> 00:10:11.240
access the R code as part of the paper?

00:10:11.690 --> 00:10:12.020
Yep.

00:10:12.305 --> 00:10:13.355
Joe Marion: It's, like plain text.

00:10:13.595 --> 00:10:14.405
It's not a package.

00:10:15.945 --> 00:10:20.300
Scott Berry: Um, uh, so by the way, the
paper is in pharmaceutical statistics,

00:10:20.600 --> 00:10:27.230
uh, came out and it's uh, uh, Barry
at also optimal sample size, division

00:10:27.230 --> 00:10:29.570
and state two stage seamless designs.

00:10:29.570 --> 00:10:32.810
If you want to grab that code,
so Lindsey, I know you have the R

00:10:32.810 --> 00:10:34.010
code right there in front of you.

00:10:34.010 --> 00:10:40.220
So if I'm gonna run a trial that
is three doses, 50, 50 50 50 in

00:10:40.220 --> 00:10:44.300
stage one, and then a hundred on
the selected dose and a hundred.

00:10:44.765 --> 00:10:45.845
On placebo.

00:10:45.845 --> 00:10:49.475
So at that point you've picked a dose
and now you're enrolling those two.

00:10:49.745 --> 00:10:55.595
What is my alpha at the end of that
trial, if I include the 50 from the dose

00:10:55.595 --> 00:11:00.905
selected and placebo from stage one with
the hundreds, I have 150 on each arm.

00:11:01.175 --> 00:11:03.485
What's the alpha that I use at the end?

00:11:04.705 --> 00:11:09.745
Lindsay Berry: Yeah, so in that example,
your information fraction is, is a third.

00:11:09.745 --> 00:11:14.035
So a third of the patients in your
analysis are coming from phase two.

00:11:14.720 --> 00:11:20.240
And in that example, your
adjusted alpha would be 0.013,

00:11:20.330 --> 00:11:25.280
essentially, so slightly
higher than the 0.0083

00:11:25.430 --> 00:11:27.860
from a Bon Ferone adjustment.

00:11:29.215 --> 00:11:29.665
Scott Berry: Okay.

00:11:29.725 --> 00:11:34.555
And so you can calculate now, you can
run the trial, select your dose, run the

00:11:34.555 --> 00:11:40.255
end of of part two, and you can run with
this adjusted alpha at the end of it.

00:11:41.030 --> 00:11:41.480
Okay.

00:11:41.930 --> 00:11:49.250
Now, uh, I, I know we, we have a picture
of what the alpha spend is a function

00:11:49.280 --> 00:11:53.870
of the information fraction, and I was
fascinated by what would that look like?

00:11:54.500 --> 00:11:55.970
Is it linear?

00:11:56.300 --> 00:11:59.450
Is it, uh, you know, I, I know is, is

00:11:59.750 --> 00:12:00.860
concave up?

00:12:00.860 --> 00:12:01.760
Do you, do you spend

00:12:01.815 --> 00:12:04.070
a, a lot of the alpha initially?

00:12:04.400 --> 00:12:11.480
Does it take to, to the end, uh, of
that, um, uh, who wants to describe these

00:12:11.480 --> 00:12:14.360
curves for people who cannot see them?

00:12:14.840 --> 00:12:15.890
Kert Viele: I nominate Lindsay.

00:12:16.055 --> 00:12:18.245
Joe Marion: feel like we
should just share it, right?

00:12:18.545 --> 00:12:19.115
Well, I mean,

00:12:19.730 --> 00:12:24.260
Scott Berry: Well, most people consume the
podcast only, you know, they're driving to

00:12:24.260 --> 00:12:24.650
work.

00:12:25.010 --> 00:12:25.430
Only

00:12:25.430 --> 00:12:27.920
about 10% of people actually see us.

00:12:28.160 --> 00:12:28.730
Um,

00:12:28.775 --> 00:12:31.265
Joe Marion: I feel like all
the best podcasts are about

00:12:31.265 --> 00:12:32.830
describing visual things too.

00:12:32.930 --> 00:12:34.790
So this is very on point for the, the

00:12:34.835 --> 00:12:35.045
Kert Viele: yeah.

00:12:35.045 --> 00:12:36.815
Don't take your eyes off the road.

00:12:36.815 --> 00:12:39.065
You know I crashed because I
was looking at an alpha chart.

00:12:39.095 --> 00:12:40.055
Not a good excuse.

00:12:41.225 --> 00:12:41.615
Scott Berry: Okay.

00:12:41.615 --> 00:12:42.725
Let me, let me ask this

00:12:42.725 --> 00:12:46.685
question, Lindsay, to describe
this if I have three doses.

00:12:46.805 --> 00:12:48.665
Uh, uh, well, actually, our, our picture

00:12:48.665 --> 00:12:48.965
doesn't

00:12:49.050 --> 00:12:50.210
Lindsay Berry: doesn't
have a three on there.

00:12:50.345 --> 00:12:52.475
Scott Berry: so let's pick two doses.

00:12:52.475 --> 00:12:53.465
Very, very simple.

00:12:53.465 --> 00:12:58.325
I have two doses in stage one, and
then I pick one and I go to stage two.

00:12:59.915 --> 00:13:04.145
W um, in that case it's, it's 0.025.

00:13:04.145 --> 00:13:10.415
If I just ran a two arm trial the whole
way at the end of the trial, it's 0.0125.

00:13:10.955 --> 00:13:12.515
Where is it halfway?

00:13:12.515 --> 00:13:17.015
Between that, what information
fraction can you, can you, can you

00:13:17.405 --> 00:13:18.755
pick up from that?

00:13:19.175 --> 00:13:22.625
So that would be, um, uh, within that.

00:13:25.655 --> 00:13:25.895
Joe Marion: Yeah.

00:13:26.930 --> 00:13:31.340
Lindsay Berry: is about an informa
information fraction of 0.2.

00:13:32.600 --> 00:13:34.010
So if you spend.

00:13:35.850 --> 00:13:38.405
Kert Viele: I, I, I thought this was
rather fascinating 'cause the first time

00:13:38.405 --> 00:13:43.475
you see this graph, your thought is, oh,
I'm spending a whole bunch of alpha with

00:13:43.505 --> 00:13:46.955
even a small exploration in phase two.

00:13:47.315 --> 00:13:53.045
And I think if you just stare at that,
your first thought is, oh no, I don't

00:13:53.045 --> 00:13:57.005
want to spend that much Alpha, maybe I
should get out of phase two really, really

00:13:57.005 --> 00:13:59.885
quickly, which isn't the right answer.

00:14:00.365 --> 00:14:00.815
So

00:14:00.965 --> 00:14:03.245
Scott Berry: no, no, you're,
you're going ahead in the paper, so

00:14:03.440 --> 00:14:05.450
Kert Viele: I know, but it's a, okay.

00:14:05.525 --> 00:14:06.420
Scott Berry: the best part of it.

00:14:06.620 --> 00:14:09.380
Lindsay Berry: Yeah, I mean, I think
what Scott's referencing here is

00:14:09.380 --> 00:14:13.640
when you look at the figure, you
see these curves representing, you

00:14:13.640 --> 00:14:17.360
know, for this number of experimental
arms, what is your alpha adjustment?

00:14:17.360 --> 00:14:22.190
And from going from zero, you're
spending zero time in phase

00:14:22.190 --> 00:14:24.560
two to 10% of your patients.

00:14:24.560 --> 00:14:26.240
In phase two, you have a huge drop.

00:14:26.825 --> 00:14:32.915
So the penalty, I'll use Scott's
favorite term penalty is incurred with

00:14:32.915 --> 00:14:35.555
very little exploration in phase two.

00:14:35.945 --> 00:14:40.145
And then with additional exploration,
the, you know, added penalty you

00:14:40.145 --> 00:14:43.535
take is, is not very much larger.

00:14:44.795 --> 00:14:45.215
Scott Berry: Oh, okay.

00:14:45.215 --> 00:14:51.605
So you said the word, um, and this
gets to the question, is it a penalty?

00:14:52.550 --> 00:14:55.460
Is this a good thing to do or not?

00:14:55.460 --> 00:15:01.100
So you have to adjust the alpha at the
end of stage two, but I get the benefit

00:15:01.100 --> 00:15:03.380
of including the data from stage one.

00:15:03.380 --> 00:15:03.410
I.

00:15:04.670 --> 00:15:06.710
But I have to adjust my alpha at the end.

00:15:06.710 --> 00:15:11.930
And, and, uh, uh, a recent podcast, I
lamented the use of the term penalty.

00:15:11.930 --> 00:15:13.700
That this is an adjustment of alpha.

00:15:13.700 --> 00:15:16.400
In many circumstances, you improve power.

00:15:16.400 --> 00:15:20.570
So it's a, it's a bad term
to, to talk about distributing

00:15:20.570 --> 00:15:22.100
or allocating your alpha.

00:15:22.580 --> 00:15:26.360
Is this a good thing to do, Lindsay?

00:15:26.600 --> 00:15:31.280
To, to adjust your alpha and
include stage one with stage two.

00:15:33.815 --> 00:15:37.265
Lindsay Berry: Yes, it is a good
thing to do in terms of having

00:15:37.265 --> 00:15:40.235
higher power than the same design.

00:15:41.315 --> 00:15:46.895
Where you only analyze stage two
data, but you get the full 0.025.

00:15:48.245 --> 00:15:53.165
Scott Berry: Okay, so we, we investigated
this question and we looked at multiple

00:15:53.165 --> 00:15:58.565
dose response, potential dose response
curve, where different number of doses,

00:15:58.565 --> 00:16:01.355
different dose response curves, and.

00:16:01.355 --> 00:16:08.465
Fair to say that in every single
scenario, it is more powerful to include

00:16:08.465 --> 00:16:11.465
stage one data, adjust your alpha.

00:16:11.645 --> 00:16:14.285
You have higher power in that scenario.

00:16:15.395 --> 00:16:16.925
Okay, so.

00:16:17.330 --> 00:16:22.850
Here we are through the paper at this
point where you can calculate with,

00:16:22.910 --> 00:16:26.510
uh, a very simple, uh, r package.

00:16:26.510 --> 00:16:29.090
It does a quadrature integration.

00:16:29.090 --> 00:16:32.150
The alpha that you have at the end,
you can write it in the protocol.

00:16:32.150 --> 00:16:35.060
At that point, it's improve power.

00:16:35.060 --> 00:16:37.970
It's a good thing to do, uh, in that.

00:16:38.180 --> 00:16:43.040
So before I pose my next question,
let me pause and make sure everybody,

00:16:43.400 --> 00:16:45.500
uh, keeps me honest where we are.

00:16:46.740 --> 00:16:50.130
Joe Marion: I'll add a little
caveat, which is that all the

00:16:50.130 --> 00:16:54.660
calculations in the paper, sort of
assume the endpoint is very quick.

00:16:55.130 --> 00:16:55.350
Scott Berry: Yep.

00:16:55.550 --> 00:16:55.850
Joe Marion: All right.

00:16:55.850 --> 00:17:00.470
That the, that the time to the endpoint
is, is fast relative to the accrual rate.

00:17:01.160 --> 00:17:07.100
And I think where this can break down a
little bit in terms of being always useful

00:17:07.400 --> 00:17:11.240
is if the endpoint is actually quite long
relative to the acre recruitment rate.

00:17:11.930 --> 00:17:16.670
Um, you end up, you know, you
don't, we don't, I think we

00:17:16.670 --> 00:17:19.220
don't really like to take breaks
between stage one and stage two.

00:17:19.220 --> 00:17:19.490
Right.

00:17:19.490 --> 00:17:20.780
Is that, is that fair Scott?

00:17:21.245 --> 00:17:21.635
Scott Berry: that's

00:17:21.635 --> 00:17:22.325
very fair.

00:17:22.565 --> 00:17:27.005
Operationally, it can be hugely
problematic if you pause enrollment

00:17:27.005 --> 00:17:28.415
at any point in the trial.

00:17:29.195 --> 00:17:29.465
Joe Marion: Yeah.

00:17:29.465 --> 00:17:35.465
And so it's um, you know, I think you
need, you need a slightly different

00:17:36.455 --> 00:17:39.395
set of calculations if you've got
one of these long-term endpoints.

00:17:39.485 --> 00:17:40.955
The math still works that we did.

00:17:40.955 --> 00:17:42.395
All, all of that is still fine.

00:17:42.395 --> 00:17:45.065
It all still plays out, but
like the relative benefit of

00:17:45.065 --> 00:17:45.930
this thing kind of changes.

00:17:46.400 --> 00:17:46.690
Yeah.

00:17:47.600 --> 00:17:51.050
Scott Berry: Yeah, so you, you, we've
made reference to the information

00:17:51.050 --> 00:17:55.310
fraction, so the proportion of patients
that are in stage one as opposed to

00:17:55.310 --> 00:17:57.740
the total, uh, size of the trial.

00:17:58.635 --> 00:18:04.995
If now you only have your primary endpoint
on a subset of stage one patients, there

00:18:04.995 --> 00:18:09.285
can be a different information for action
calculated, uh, within that setting.

00:18:09.285 --> 00:18:12.105
And some patients in stage one
might not actually contribute to

00:18:12.105 --> 00:18:13.965
that until later in the trial.

00:18:14.325 --> 00:18:16.575
So this is something we do quite a bit.

00:18:16.575 --> 00:18:20.505
We might employ simulation, we might
just be able to calculate the information

00:18:20.505 --> 00:18:25.155
fraction and then evaluating whether
this is a positive thing to do and

00:18:25.155 --> 00:18:26.925
the difference in the sample size.

00:18:27.225 --> 00:18:30.165
There's, there's complexity
to many of these trials.

00:18:30.165 --> 00:18:31.150
I think that's very fair.

00:18:32.215 --> 00:18:32.545
Joe Marion: Yeah.

00:18:32.575 --> 00:18:34.375
I think the other way to
think about it is like.

00:18:34.970 --> 00:18:38.060
You know, you might, you're
over-enrolled in your phase, in

00:18:38.060 --> 00:18:39.560
your stage one in some sense, right?

00:18:39.560 --> 00:18:42.860
You have 25 patients who've
completed the endpoint, but maybe

00:18:42.860 --> 00:18:44.630
you have another 25 outstanding.

00:18:44.960 --> 00:18:47.570
They're not contributing
any data to the analysis.

00:18:47.600 --> 00:18:52.820
You don't pay any penalty for not
having their data, but because,

00:18:52.820 --> 00:18:54.410
um, but you've still enrolled them.

00:18:54.710 --> 00:18:54.950
Right?

00:18:54.950 --> 00:18:57.710
And so in some sense, making
stage one bigger is sort of

00:18:57.710 --> 00:18:59.450
extra expensive in that setting.

00:18:59.780 --> 00:18:59.900
Yeah.

00:19:00.290 --> 00:19:04.670
Scott Berry: Yeah, I, I recently
did a project where it was a client

00:19:04.670 --> 00:19:06.980
had two doses in a, in a control.

00:19:07.490 --> 00:19:12.140
In stage one, they were picking one
of the two doses moving to equal

00:19:12.140 --> 00:19:19.250
randomization, one-to-one in, in stage
two, and the, they, they had this

00:19:19.250 --> 00:19:21.770
aspect that the endpoint was 12 weeks.

00:19:22.625 --> 00:19:25.865
And so if they enrolled stage
one, they'd have, uh, data.

00:19:25.865 --> 00:19:29.195
At 12 weeks there was
gonna be over enrolling.

00:19:29.195 --> 00:19:33.605
But with two doses, suppose you over
enrolled 10 patients on each of the arms.

00:19:34.415 --> 00:19:37.985
You've 20 of those patients
count in phase three.

00:19:39.020 --> 00:19:44.210
And so in some sense, 10 patients are
quote unquote wasted with within that.

00:19:44.630 --> 00:19:48.230
And of course it depends on the
length of enrollment and, and, and

00:19:48.230 --> 00:19:52.580
the time to that which is evaluating
each one of these trial designs.

00:19:52.850 --> 00:19:56.060
Now, the other interesting part of
it was they actually had earlier

00:19:56.060 --> 00:20:00.020
information than 12 weeks, which was
very highly correlated to 12 weeks.

00:20:00.020 --> 00:20:04.640
And we build that in with a longitudinal
model to make that, make that decision.

00:20:05.465 --> 00:20:05.685
Joe Marion: Yep.

00:20:06.410 --> 00:20:07.460
Scott Berry: Okay, so.

00:20:08.300 --> 00:20:11.120
Now getting to where,
where Kurt wanted to go.

00:20:11.480 --> 00:20:14.150
So seamless.

00:20:14.150 --> 00:20:15.710
Two three top trial.

00:20:16.400 --> 00:20:19.550
You can calculate the alpha
very straightforwardly.

00:20:20.450 --> 00:20:21.740
It's a good thing to do.

00:20:21.800 --> 00:20:25.310
You want to include stage one
data as part of stage two.

00:20:26.330 --> 00:20:32.990
What that mean about the size of the
phase two part, stage one of the trial

00:20:33.560 --> 00:20:36.500
relative to doing separate trials?

00:20:36.590 --> 00:20:36.980
Kurt?

00:20:38.145 --> 00:20:39.015
Kert Viele: Oh, I get called on.

00:20:39.405 --> 00:20:43.185
Um, so one of the things that
was interesting, so we explored.

00:20:43.565 --> 00:20:45.575
Uh, a lot of different dose responses.

00:20:45.575 --> 00:20:49.775
So some dose responses where a
couple doses are good and the

00:20:49.775 --> 00:20:55.325
rest of the doses are bad dose
responses that had a linear increase.

00:20:55.505 --> 00:20:57.155
Um, different kinds of shapes.

00:20:57.725 --> 00:21:02.795
And one thing that's was really
interesting for the separate trials is.

00:21:03.270 --> 00:21:09.270
Depending on how the doses actually stack
up, you wanna do very different splits

00:21:09.420 --> 00:21:14.400
in how much of your total development
is in phase two versus phase three.

00:21:14.580 --> 00:21:20.250
You're really at risk of doing
too few or too many if you don't

00:21:20.250 --> 00:21:21.720
get that dose response right?

00:21:21.720 --> 00:21:24.690
If there's a different number
of doses that are good or bad.

00:21:25.700 --> 00:21:30.020
On the other hand, if you do the
seamless, you've got the option where

00:21:30.020 --> 00:21:32.420
you get to keep that phase two data.

00:21:32.600 --> 00:21:35.990
So going a little bit longer,
you're not, you know, you, you do

00:21:35.990 --> 00:21:38.120
lose something by going farther.

00:21:38.120 --> 00:21:43.400
In phase two, you don't focus as much,
but it's a lot less than separate trials.

00:21:43.640 --> 00:21:47.150
So it became a very robust decision that.

00:21:47.570 --> 00:21:52.850
Somewhere in the 30 to 40% of your
total development ought to be spent

00:21:52.910 --> 00:21:58.790
in phase two in that setting, and it
was very robust across dose responses.

00:21:59.000 --> 00:22:03.140
It really helped people out in
sizing the trials without really

00:22:03.140 --> 00:22:04.460
having to think about it too much.

00:22:06.905 --> 00:22:11.765
Scott Berry: Oh, and in every single
scenario, the size of phase two,

00:22:12.185 --> 00:22:15.935
number of doses, dose response,
the truth of the dose response,

00:22:16.445 --> 00:22:19.745
fa, the phase two part is bigger.

00:22:19.805 --> 00:22:21.395
If you do seamless trials.

00:22:22.970 --> 00:22:28.700
Lindsay Berry: Yeah, the way we approached
this was we identified for the seamless

00:22:28.850 --> 00:22:36.170
development program and the separate
development program, which, um, how,

00:22:36.200 --> 00:22:41.660
what is the optimal way to allocate your
total number of patients between stages

00:22:41.660 --> 00:22:44.900
in terms of, uh, um, maximizing power.

00:22:45.350 --> 00:22:48.350
So, um, you're right, yeah.

00:22:48.350 --> 00:22:49.820
In the seamless designs.

00:22:50.390 --> 00:22:55.640
The way to allocate your patients was
to put more in phase two to optimize

00:22:55.640 --> 00:22:59.210
power compared to the separate designs.

00:23:00.965 --> 00:23:07.355
Scott Berry: So another part of
that is that by having a larger

00:23:07.355 --> 00:23:12.245
phase two, you're more likely to
pick the better arm in the setting.

00:23:13.955 --> 00:23:14.045
Okay, so.

00:23:14.870 --> 00:23:17.510
Joe Marion: And you get to keep
the data from that arm, right?

00:23:17.510 --> 00:23:20.810
So if you're worried about like
penalties, you're actually like,

00:23:20.960 --> 00:23:23.690
you know, you're starting with
your best, your best data set.

00:23:24.105 --> 00:23:27.245
Rather than having to start
all the way over again, so

00:23:28.310 --> 00:23:31.520
Scott Berry: so, um,
easy to calculate Alpha.

00:23:32.480 --> 00:23:33.860
Phase two is bigger.

00:23:33.860 --> 00:23:38.570
We're more likely to get the better dose,
which by the way doesn't just affect

00:23:38.570 --> 00:23:47.600
stage two and power, but the eventual
approval of the drug, the dose used that.

00:23:47.600 --> 00:23:50.390
This is a thing that I
think everybody would want.

00:23:50.655 --> 00:23:51.225
Is bigger.

00:23:51.225 --> 00:23:55.395
Phase two, when you run separate
trials, it's harder because you gotta

00:23:55.395 --> 00:23:58.395
get out and you gotta get to phase
three 'cause patent life is going.

00:23:59.025 --> 00:24:02.775
But here, when those patients included
in phase three, and by the way,

00:24:02.775 --> 00:24:06.885
you're losing the white space between
trials, huge advantages of this.

00:24:06.885 --> 00:24:10.095
You're also getting a better
dose, which, which is good.

00:24:10.845 --> 00:24:18.615
There's a lot of win, win, win in the
seamless two three trial design scenario.

00:24:22.325 --> 00:24:23.015
Okay.

00:24:23.405 --> 00:24:30.215
Now returning to an initial question
where I, all I said was, you know, this

00:24:30.215 --> 00:24:33.845
closed testing procedure is awkward.

00:24:35.015 --> 00:24:42.755
Uh, within it, did we, we compared
the group sequential alpha calculation

00:24:43.055 --> 00:24:45.275
to the closed testing procedure.

00:24:45.275 --> 00:24:47.645
And what did we find, Joe?

00:24:48.560 --> 00:24:49.310
Joe Marion: I actually.

00:24:50.615 --> 00:24:51.275
Lindsay,

00:24:53.135 --> 00:24:53.825
Scott Berry: I, yeah.

00:24:53.970 --> 00:24:55.110
Lindsay Berry: Yeah, we did.

00:24:55.170 --> 00:24:56.430
So we added,

00:24:57.040 --> 00:24:58.095
Kert Viele: We as said, Lindsay did.

00:24:59.705 --> 00:25:00.155
Scott Berry: Yeah.

00:25:00.245 --> 00:25:02.675
Joe Marion: I think we looked up
some of the math, but, but yeah.

00:25:02.675 --> 00:25:03.125
Lindsay, yeah.

00:25:03.740 --> 00:25:06.560
Lindsay Berry: Yeah, so we did,
we implemented the closed testing

00:25:06.590 --> 00:25:09.650
procedure in one specific way.

00:25:09.650 --> 00:25:13.100
And I wanna say that there's,
there's probably many ways you

00:25:13.100 --> 00:25:16.250
could, you know, figure out the
details of how to implement this.

00:25:16.250 --> 00:25:20.660
So this is just what we found,
um, in our implementation.

00:25:21.410 --> 00:25:27.890
Um, the results overall were similar,
um, but in some scenarios, the.

00:25:28.520 --> 00:25:33.290
Closed testing procedure had lower
power than the seamless design with

00:25:33.290 --> 00:25:37.040
this group sequential math that,
that Joe explained at the beginning.

00:25:37.430 --> 00:25:43.310
And it was particularly evident in
scenarios when one or very few doses

00:25:43.310 --> 00:25:47.930
had an effect, you'd see the power
of the closed testing procedure drop.

00:25:48.380 --> 00:25:49.880
Um, and I think that's just the.

00:25:50.990 --> 00:25:54.860
Uh, a reflection of how the closed
testing procedure works, where you

00:25:54.860 --> 00:26:00.500
have to test other hypotheses first
before you can reject the null

00:26:00.500 --> 00:26:02.750
hypothesis on the selected arm.

00:26:02.990 --> 00:26:08.630
So that sort of blocks, um, the ability
to reject the null hypothesis and

00:26:08.630 --> 00:26:12.950
lowers your power in this scenario
where only one dose actually works.

00:26:14.720 --> 00:26:17.690
Scott Berry: So it was a bit
conservative in type one error

00:26:17.690 --> 00:26:19.190
in some of these scenarios.

00:26:19.400 --> 00:26:26.270
It had lower power in there, so also it,
it, it does very well in terms of power

00:26:26.270 --> 00:26:31.670
and, and better than some of the, the,
the general closed testing procedure, a

00:26:31.670 --> 00:26:32.510
aspects of it.

00:26:33.245 --> 00:26:35.555
Joe Marion: there's a weird thing
about the closed testing procedure

00:26:35.555 --> 00:26:39.785
too, where, I mean, like, what is your
estimate or your, your estimate, right?

00:26:39.785 --> 00:26:44.915
Like, you know, it's, I, I guess it's
probably the mean from the pooled

00:26:44.915 --> 00:26:48.995
stages, but you don't, that's not
actually where you got your p-value from.

00:26:49.475 --> 00:26:53.225
Um, I assume there's no, like,
assume it's, you know, you're

00:26:53.225 --> 00:26:57.155
unlikely to have like a weird case
where your pooled analysis doesn't,

00:26:57.245 --> 00:26:58.715
you know, meet significance.

00:26:58.715 --> 00:27:00.425
But it's just kind of weird
to me to have your like.

00:27:01.235 --> 00:27:04.925
Your test and your model kind of
divorced from each other in this way?

00:27:05.315 --> 00:27:05.495
Yeah.

00:27:06.815 --> 00:27:08.135
Scott Berry: So that brings out.

00:27:08.810 --> 00:27:14.210
In number of designs we do, people working
on phase two, three seamless trials.

00:27:14.390 --> 00:27:17.690
A good number of them
fit into this bucket.

00:27:17.690 --> 00:27:19.790
We put things and we made it very simple.

00:27:20.510 --> 00:27:26.870
Fix sample size, stage one, fix sample
size, stage two, select a dose, and

00:27:26.870 --> 00:27:33.140
move forward a number of the, the, these
designs end up with innovations to them.

00:27:33.500 --> 00:27:38.300
Adaptive things happening in,
in the stage one part, you could

00:27:38.300 --> 00:27:40.850
potentially drop some doses sooner.

00:27:40.850 --> 00:27:44.540
You could potentially
do RAR on those doses.

00:27:44.930 --> 00:27:51.950
You could have adaptive sample size
of stage two part, and then in the

00:27:51.950 --> 00:27:54.470
stage, that was the stage one part.

00:27:54.470 --> 00:27:56.900
I knew I would mess up
phase two and stage one.

00:27:57.410 --> 00:28:00.530
Um, so stage one can
have various adaptations.

00:28:01.730 --> 00:28:02.900
Stage two.

00:28:03.200 --> 00:28:06.980
Can have adaptations mostly
probably in sample size.

00:28:06.980 --> 00:28:08.840
Maybe you could stop enrolling sooner.

00:28:08.840 --> 00:28:13.280
The effect is larger, flexible
sample size within those parts.

00:28:13.580 --> 00:28:19.580
So that, that's where this gets kind
of neat to, to, to use this and add

00:28:20.570 --> 00:28:26.930
adaptations, uh, uh, to both parts
of this and within seamless trials.

00:28:28.985 --> 00:28:30.695
Kert Viele: So, you know,
there's a couple parts to that.

00:28:30.695 --> 00:28:34.955
You know, one, one question is, well,
if you're gonna be complicated, what?

00:28:34.985 --> 00:28:36.875
Why worry about this result?

00:28:36.875 --> 00:28:38.645
You're gonna have to
simulate everything I.

00:28:39.080 --> 00:28:42.800
But I do think we have a lot of
clients who come to us and that

00:28:42.830 --> 00:28:47.000
one of their questions is, should
I even pursue a seamless trial?

00:28:47.300 --> 00:28:50.660
And the ability to go,
Hey, here's the basics.

00:28:50.690 --> 00:28:53.780
Here's a simple one, and
here's what you gain.

00:28:54.230 --> 00:28:57.830
It's worth making that kind of
broad discussion over whether it's

00:28:57.830 --> 00:28:59.445
worth investigating things further.

00:28:59.755 --> 00:29:03.530
Usually the answer is yes, it's
worth investigating things further.

00:29:04.030 --> 00:29:07.270
So, but that's certainly there,
there's value on both sides of that.

00:29:07.270 --> 00:29:10.510
Even if you move away from where
the results directly applicable,

00:29:12.060 --> 00:29:14.280
Joe Marion: Yeah, I, I think there
are also a lot of interesting

00:29:14.280 --> 00:29:17.490
situations too, where the result
is like conservative and so you can

00:29:17.490 --> 00:29:20.730
still apply it even though you're not
really following the like underlying.

00:29:21.335 --> 00:29:23.285
Like procedure that's
described in the paper.

00:29:23.285 --> 00:29:27.275
So, uh, Scott was talking about having
a longitudinal model earlier, right?

00:29:27.275 --> 00:29:30.275
To use your early week data, you
know, your week 12, to predict

00:29:30.275 --> 00:29:32.015
your week 52, for example.

00:29:32.705 --> 00:29:37.025
And you, you can apply this model in
that s you can apply this correction.

00:29:37.025 --> 00:29:42.365
In that situation, what you do is you,
you treat the information, you calculate

00:29:42.365 --> 00:29:45.425
the information infraction, assuming
you had all the data at the time of the

00:29:45.425 --> 00:29:47.795
interim, and that'll give you a penalty.

00:29:47.795 --> 00:29:48.995
That's kind of too big.

00:29:49.535 --> 00:29:51.485
But it is conservative.

00:29:51.485 --> 00:29:53.255
It gives you analytic
control type one error.

00:29:53.255 --> 00:29:55.715
You don't have to simulate
if you don't want to.

00:29:56.075 --> 00:29:59.045
And honestly, I found that that
works really well and is like.

00:29:59.405 --> 00:30:02.795
Absolutely acceptable by
regulators also applies to like

00:30:02.795 --> 00:30:04.355
the besian models we do, right?

00:30:04.445 --> 00:30:08.225
Um, you may have a dose selection
criteria coming from sort of

00:30:08.225 --> 00:30:09.905
a complicated besian model.

00:30:10.115 --> 00:30:13.295
You may not be choosing the best
dose, you may be choosing the ED 90.

00:30:13.745 --> 00:30:17.345
You can still apply this procedure
in those settings and it will be

00:30:17.375 --> 00:30:20.075
conservative and that's really helpful.

00:30:20.585 --> 00:30:24.275
Um, I'm always happy when I can
move past like simulation based

00:30:24.275 --> 00:30:25.475
control type one error, right?

00:30:25.475 --> 00:30:26.075
And into.

00:30:27.020 --> 00:30:29.300
And, and have that part of the,
the puzzle solved so I can focus

00:30:29.300 --> 00:30:30.495
on the other pieces of the design.

00:30:31.430 --> 00:30:31.580
Scott Berry: Yeah.

00:30:32.330 --> 00:30:38.540
So in in the scenarios, you might want
to pick the dose that that is, is optimal

00:30:38.540 --> 00:30:40.850
over tolerability and safety at the same

00:30:40.850 --> 00:30:41.180
time.

00:30:41.180 --> 00:30:45.830
But the selection of any dose would be
controlled under this procedure would only

00:30:45.830 --> 00:30:49.280
be cases such as we're gonna take the.

00:30:49.535 --> 00:30:51.335
Uh, middle dose, no.

00:30:51.815 --> 00:30:56.945
As long as it's, or sorry, the
high dose, unless it's unsafe and

00:30:56.945 --> 00:31:00.725
if based on safety, it's unsafe,
we're gonna take the lower dose.

00:31:01.175 --> 00:31:04.145
That's a case where you probably
don't even have to split alpha in a

00:31:04.145 --> 00:31:05.225
circumstances like that.

00:31:05.225 --> 00:31:09.485
It's not efficacy that's selecting dose,
but largely in the other circumstances,

00:31:09.485 --> 00:31:13.055
we, we've been able to, to, I employ that.

00:31:14.070 --> 00:31:20.760
So not every scenario should do a seamless
two three trial, though the, the math

00:31:20.760 --> 00:31:25.170
is great and almost, you should almost
have to figure out why you're not doing

00:31:25.170 --> 00:31:31.110
it, but what are some areas where it
might not be the right thing for them

00:31:31.110 --> 00:31:33.510
to do, to do Seamless two, three trials.

00:31:35.195 --> 00:31:38.225
Kert Viele: You have to use different
endpoints in phase two and phase three.

00:31:38.285 --> 00:31:40.415
We don't like it, but it would break this.

00:31:42.320 --> 00:31:46.010
Scott Berry: So one scenario might be
that it's a relatively new disease,

00:31:46.010 --> 00:31:50.630
not a rich history of knowing even
what the phase three endpoint would be.

00:31:50.870 --> 00:31:54.950
Sometimes the FDA says, bring us the phase
two and we'll figure out the endpoint.

00:31:54.950 --> 00:31:58.730
It would be hard to have a seamless
trial, uh uh, in that setting.

00:31:58.970 --> 00:32:03.500
Not impossible, but that might be a hard
setting where the regulatory pathway of

00:32:03.500 --> 00:32:09.950
phase three is so unclear that you kind
of need phase two to to do that is is one

00:32:10.340 --> 00:32:11.150
scenario

00:32:11.785 --> 00:32:12.565
Lindsay Berry: yeah, I think

00:32:13.415 --> 00:32:13.535
Scott Berry: I.

00:32:13.705 --> 00:32:16.705
Lindsay Berry: in general, to do a
seamless two, three trial, you need

00:32:16.705 --> 00:32:20.815
to, before the trial when you're
designing it, be able to commit to.

00:32:21.470 --> 00:32:25.400
A method for selecting a dose,
so an endpoint and you know

00:32:25.400 --> 00:32:26.930
which dose you wanna select.

00:32:26.930 --> 00:32:31.430
And sometimes it might not be, you
might not know how to do that yet,

00:32:31.550 --> 00:32:35.840
and you might wanna see all of the
data and look at all of the endpoints

00:32:35.840 --> 00:32:38.960
and kind of play around in the data
and pick the endpoint yourself.

00:32:39.680 --> 00:32:43.550
So I think that might be a case where
being able to look at the phase two

00:32:43.550 --> 00:32:48.290
data before you design your phase
three trial, uh, might be preferable.

00:32:49.625 --> 00:32:51.020
Kert Viele: But I think
it's always good to.

00:32:51.170 --> 00:32:51.770
Scott Berry: where.

00:32:52.130 --> 00:32:56.540
I might try to push back and say,
okay, you, you're losing 200 patients.

00:32:56.990 --> 00:32:59.960
Uh, you know, and think hard
about the dis decisions.

00:32:59.960 --> 00:33:03.290
But, but there are some circumstances
where it, it's really hard.

00:33:04.160 --> 00:33:09.110
Uh, another circumstances may
involve just CMC issues that

00:33:09.410 --> 00:33:15.890
the, the method for creating the
drug is not phase three ready.

00:33:16.700 --> 00:33:19.940
And the FDA wants you
to be using the drug.

00:33:19.940 --> 00:33:21.410
That's phase three ready.

00:33:21.410 --> 00:33:26.630
The manufacturing is set up in a way
and so that might be a case where you

00:33:26.630 --> 00:33:31.400
can't use the phase two data because
the manufacturing isn't even part of it.

00:33:31.580 --> 00:33:33.590
You do operationally seamless potentially.

00:33:33.590 --> 00:33:36.290
There, there, there's a, but that's
a scenario where I've seen that

00:33:36.290 --> 00:33:37.850
that's a bit, a bit of a hurdle.

00:33:38.270 --> 00:33:38.630
The other

00:33:38.630 --> 00:33:40.280
one I was spend a little bit of time on.

00:33:40.280 --> 00:33:40.760
Go ahead Joe.

00:33:41.265 --> 00:33:44.925
Joe Marion: Scott, is there a
situation where you have approval to

00:33:44.925 --> 00:33:50.055
like study the drug over say 26 weeks
of exposure, something like that.

00:33:50.055 --> 00:33:52.515
But the FDA hasn't approved you to like.

00:33:52.895 --> 00:33:57.095
Look at 52 weeks or 104 weeks, your
actual primary endpoint is, and so

00:33:57.095 --> 00:34:00.605
you really can't even collect the
data you need in the phase two.

00:34:01.310 --> 00:34:01.700
Scott Berry: Yeah.

00:34:01.700 --> 00:34:06.860
and, and it might be you don't have,
uh, you don't have the data enough to

00:34:06.860 --> 00:34:09.080
give a year of exposure, but you can do.

00:34:09.980 --> 00:34:11.660
Uh, you know, 12 weeks of exposure,

00:34:11.990 --> 00:34:13.250
but your endpoint is 12

00:34:13.325 --> 00:34:13.955
Joe Marion: step.

00:34:14.315 --> 00:34:14.645
Yeah.

00:34:14.780 --> 00:34:20.060
Scott Berry: Uh, or you don't want to
put in the resources to do a 12 month

00:34:20.060 --> 00:34:27.260
study before you get proof of concept in
sort of a, a, a, a phase two trial, and

00:34:27.260 --> 00:34:29.000
you're only gonna get 12 weeks exposure.

00:34:29.540 --> 00:34:33.200
Uh, might be a scenario, um, in
that, by the way, all of these

00:34:33.200 --> 00:34:35.360
trials seamless, two three trials.

00:34:35.720 --> 00:34:38.240
That we do, you put a proof of concept.

00:34:38.270 --> 00:34:42.110
You, you, you, you need to jump
a particular hurdle of reasonable

00:34:42.110 --> 00:34:46.100
efficacy or you don't even enact
the, the, the stage two part.

00:34:46.100 --> 00:34:49.340
So you can build in proof
of concept as part of this.

00:34:49.850 --> 00:34:54.200
And that brings me to the, the, the
other one that we do a fair amount

00:34:54.200 --> 00:35:00.200
of work with smaller biotechs and
a funding becomes a big issue.

00:35:00.620 --> 00:35:05.300
And the, the notion is that
we, we don't have the money.

00:35:05.735 --> 00:35:09.605
To run the phase three part,
we need the phase two data

00:35:09.605 --> 00:35:11.765
to raise the money for phase.

00:35:12.395 --> 00:35:12.815
Three.

00:35:13.145 --> 00:35:15.275
We need that phase two data for that.

00:35:15.425 --> 00:35:20.735
And so we need to make that data public
and go to a conference and say, look

00:35:20.735 --> 00:35:23.645
how good the data are in that setting.

00:35:23.975 --> 00:35:28.205
And if it's part of a phase three
trial and seamless phase two three,

00:35:28.205 --> 00:35:32.375
that, that's generally frowned upon
that you unblind data as part of that.

00:35:32.615 --> 00:35:35.165
So that could be a potential restriction.

00:35:35.765 --> 00:35:36.875
It's something we've

00:35:36.910 --> 00:35:37.270
Kert Viele: Scott.

00:35:37.270 --> 00:35:41.530
this would, that would apply to NIH
funding as well, where you need the

00:35:41.530 --> 00:35:43.720
phase two data to get the grant for phase

00:35:43.925 --> 00:35:44.285
Scott Berry: Yep.

00:35:45.845 --> 00:35:52.985
Now we do a lot of work with biotechs
that we try to set up objective criteria

00:35:52.985 --> 00:35:59.180
in phase two, that it won't go to phase
three unless a certain effect size

00:35:59.200 --> 00:36:05.315
is seen, a probability of clinical
significant effect sizes seen, and you

00:36:05.315 --> 00:36:07.505
could potentially set up with a funder.

00:36:07.850 --> 00:36:12.350
That if that comes back positive and
all you get back is, did we achieve

00:36:12.710 --> 00:36:14.540
the proof of concept threshold?

00:36:14.540 --> 00:36:15.320
Yes or no?

00:36:15.965 --> 00:36:20.555
And that's done by an independent
stack group, and they provide that.

00:36:20.555 --> 00:36:24.845
The funder says, yes, if you can hit
that we'll, we'll, we'll help fund it.

00:36:25.445 --> 00:36:30.485
The benefits are smaller,
sample size, shorter timeline.

00:36:30.485 --> 00:36:34.595
It's good for the funders in that
circumstance and just for the whole

00:36:34.595 --> 00:36:36.575
reasons that this is a better approach.

00:36:36.815 --> 00:36:41.195
So we've done a good bit of that, but that
could be a constraint in a number of areas

00:36:41.195 --> 00:36:44.135
where largely regulators think that that.

00:36:44.290 --> 00:36:48.430
Part one is really part of a phase
three trial 'cause the patients

00:36:48.430 --> 00:36:52.030
are so that could be a potential
constraint in all of this.

00:36:53.705 --> 00:36:57.545
Joe Marion: those, those proof of concept
thresholds are really important, but

00:36:57.545 --> 00:37:01.685
they're tricky because they kind of change
the way the trial looks in a lot of ways.

00:37:02.435 --> 00:37:05.645
If there's a pretty high hurdle, you
have to jump from phase, from stage one

00:37:05.645 --> 00:37:09.815
to stage two, that's gonna reduce, you
know, the thing you, you normally would

00:37:09.815 --> 00:37:14.255
describe as like the power of the design
because, you know, to win the design, it's

00:37:14.255 --> 00:37:19.115
not just meeting a significant threshold
at the end of it, but it's also jumping

00:37:19.115 --> 00:37:20.495
over that hurdle in the middle of it.

00:37:21.155 --> 00:37:24.815
And I think that's, this is like
always a part of clinical trial design.

00:37:24.845 --> 00:37:28.085
It's like part of every drug
development program, but it, it's

00:37:28.085 --> 00:37:31.175
kind of hidden if you don't do a
seamless design because no one.

00:37:31.670 --> 00:37:34.280
Specifies really what that hurdle is.

00:37:34.550 --> 00:37:35.720
You don't ever see it.

00:37:35.810 --> 00:37:38.780
You know, people are always
calculating power for their phase

00:37:38.780 --> 00:37:40.700
three after they've hit the hurdle.

00:37:41.420 --> 00:37:44.660
Um, and so it, it's kind of tough
to compare these, this is a really

00:37:44.660 --> 00:37:49.130
important part of doing these designs,
but it, it can lead to some comparisons

00:37:49.130 --> 00:37:50.810
that people find confusing, I think.

00:37:51.020 --> 00:37:51.200
Yeah.

00:37:52.595 --> 00:37:52.865
Scott Berry: Yeah.

00:37:52.865 --> 00:37:58.145
Some of them are designed in a way that
there is a pretty high hurdle, but they're

00:37:58.145 --> 00:38:03.275
sort of built like, boy if, but if we hit
that hurdle, we want full acceleration.

00:38:03.275 --> 00:38:04.145
We want full goal.

00:38:04.145 --> 00:38:06.695
If we don't hit that big
hurdle, we're happy to just

00:38:06.695 --> 00:38:08.075
have this be a phase two trial.

00:38:08.765 --> 00:38:13.145
And we'll analyze the data and we
might go run a phase three program.

00:38:13.625 --> 00:38:17.465
Um, and others are set up where
it's a relatively low hurdle.

00:38:17.465 --> 00:38:21.815
They're, they're gonna go to phase
three most of the time, uh, and set

00:38:21.815 --> 00:38:23.345
it up almost like a futility rule.

00:38:23.345 --> 00:38:26.465
So we've been involved
the whole gamut of that.

00:38:26.525 --> 00:38:30.155
Um, uh, which can be customized
to the particular scenario.

00:38:30.155 --> 00:38:30.935
The person's in.

00:38:31.685 --> 00:38:33.875
Joe Marion: Yeah, the, the high
hurdle I think of is like a

00:38:33.875 --> 00:38:36.515
design that's opportunity seeking.

00:38:37.085 --> 00:38:40.535
It's like designed to go really fast
if things look well, and otherwise it

00:38:40.535 --> 00:38:42.425
just turns into your normal phase two.

00:38:43.100 --> 00:38:43.460
Scott Berry: Yep,

00:38:43.475 --> 00:38:46.715
Joe Marion: Um, but you know, you
gotta get in that mindset and you

00:38:46.715 --> 00:38:49.655
gotta convince other people that,
that that's the right mindset to have.

00:38:50.510 --> 00:38:50.660
Scott Berry: yep.

00:38:51.500 --> 00:38:51.740
Yep.

00:38:54.920 --> 00:38:55.790
All right.

00:38:55.850 --> 00:38:58.910
Any other thoughts on this?

00:38:58.910 --> 00:39:02.720
This paper, it was one of my favorite
papers in, in terms of setting this

00:39:02.720 --> 00:39:06.350
up, making this straightforward, but
the optimality part of it I thought

00:39:06.350 --> 00:39:08.090
was just absolutely fascinating.

00:39:08.150 --> 00:39:10.850
And the ramifications to
drug development also.

00:39:11.270 --> 00:39:13.010
Uh, absolutely fascinating.

00:39:14.060 --> 00:39:15.860
Joe Marion: Yeah, this is
one of the ones we use a lot.

00:39:15.950 --> 00:39:19.700
Like I, I use this regularly in our
project, so that, that's been cool.

00:39:20.090 --> 00:39:20.300
Scott Berry: Yep.

00:39:21.715 --> 00:39:24.175
Lindsay Berry: Yeah, there's,
yeah, there's R Code.

00:39:24.205 --> 00:39:29.005
Joe's really nice R code in the
appendix, so super easy to use.

00:39:29.005 --> 00:39:33.025
Just plug it in and then you
can calculate it in seconds.

00:39:33.055 --> 00:39:33.355
Uh.

00:39:33.355 --> 00:39:37.075
Um, or you can just do what
Scott does and you can ask

00:39:37.310 --> 00:39:39.225
Joe Marion: right seconds
would be really long.

00:39:40.705 --> 00:39:43.705
Lindsay Berry: I was gonna say, or you can
do what Scott does and you can message me

00:39:43.705 --> 00:39:45.505
and Joe and we'll calculate it for you.

00:39:45.565 --> 00:39:46.640
So that's also an option.

00:39:47.675 --> 00:39:50.015
Scott Berry: uh, and they have
passed me the code and I have

00:39:50.015 --> 00:39:51.335
actually learned to run it.

00:39:51.335 --> 00:39:53.915
So if I can do it, all of you can do it.

00:39:53.945 --> 00:39:56.855
Joe Marion: Scott's just waiting for
the FORTRAN version of it, right?

00:39:56.885 --> 00:39:58.505
Like that's, that's what we need.

00:39:58.595 --> 00:39:59.765
Scott Berry: It will be faster.

00:39:59.765 --> 00:40:02.855
But this code is so fast
that you wouldn't notice.

00:40:02.915 --> 00:40:04.530
Um, within them, I'm sure.

00:40:05.255 --> 00:40:07.235
Joe Marion: What do you
think seamless means?

00:40:07.265 --> 00:40:08.585
Like, I think of shirts.

00:40:08.645 --> 00:40:12.755
Is that, what is that the seam
like, you know, it's, you can't

00:40:12.755 --> 00:40:14.980
tell where the sleeve connects to
the body or is it something else?

00:40:15.590 --> 00:40:20.390
Scott Berry: I, I think you can't tell
where the stage one and the stage two

00:40:20.390 --> 00:40:23.210
part, there's no seam between them.

00:40:23.570 --> 00:40:25.670
There's no white space, which is a killer

00:40:26.030 --> 00:40:27.200
in drug development.

00:40:27.200 --> 00:40:31.640
So we, maybe we could call
these white space free designs.

00:40:32.660 --> 00:40:32.750
Joe Marion: I.

00:40:32.750 --> 00:40:33.290
like it.

00:40:34.815 --> 00:40:35.750
Lindsay Berry: Trademark that.

00:40:36.650 --> 00:40:38.030
Joe Marion: we should talk about blinding.

00:40:38.120 --> 00:40:39.560
'cause that's something that comes up too.

00:40:40.895 --> 00:40:45.275
Um, like I've had people claim that
you can't do these designs because

00:40:45.275 --> 00:40:49.055
the decision to go from, I've had,
I've had people claim that the sponsor

00:40:49.055 --> 00:40:55.655
isn't allowed to know when the design
enters stage two, which is not true.

00:40:55.685 --> 00:40:56.615
I think it's crazy.

00:40:56.615 --> 00:40:59.045
I, is that, is that the consensus?

00:40:59.645 --> 00:41:00.575
Kert Viele: We certainly have run 'em.

00:41:00.650 --> 00:41:01.010
Scott Berry: right.

00:41:01.715 --> 00:41:05.285
Kert Viele: So we've, we've
run them at least a decade.

00:41:05.285 --> 00:41:05.975
Right Scott?

00:41:06.475 --> 00:41:06.765
Scott Berry: Yeah.

00:41:06.770 --> 00:41:06.890
Yeah.

00:41:06.890 --> 00:41:06.980
And,

00:41:07.820 --> 00:41:10.730
and they, there, there's actually
a fair amount of information.

00:41:10.730 --> 00:41:16.160
I think what gets everybody nervous is if
you were to unblind any individual patient

00:41:16.340 --> 00:41:20.960
and you know, John Doe is on placebo
or not, that this can be problematic.

00:41:20.960 --> 00:41:21.410
I mean, you, you.

00:41:21.935 --> 00:41:26.285
They, they, they still have outcomes
to in the trial or if you know

00:41:26.285 --> 00:41:30.425
exactly the effect size, there's
concerns about operational bias.

00:41:30.635 --> 00:41:34.715
And, and the FDA guidance on
adaptive trial spends about half

00:41:34.715 --> 00:41:36.515
of its time on operational bias.

00:41:36.515 --> 00:41:38.375
Does it change the patient's enrolled?

00:41:38.615 --> 00:41:43.625
Do the clinical sites, uh, uh, differ
in the different parts of the trial?

00:41:43.865 --> 00:41:45.635
But you can get a certain information.

00:41:45.635 --> 00:41:47.405
You know, you hit proof of concept.

00:41:48.050 --> 00:41:52.880
Uh, you know that the effect was
above, uh, that, and you know,

00:41:52.880 --> 00:41:54.800
what doses selected even in

00:41:54.800 --> 00:41:58.940
part two in that, and it might
spawn another phase three trial.

00:41:59.150 --> 00:42:02.750
So this might be the first of your two
adequate and well controlled, and now you

00:42:02.750 --> 00:42:08.030
need to know the dose for the other phase
three trial, even sizing the other trial.

00:42:08.150 --> 00:42:13.550
So there's a fair amount that gets known,
but there is a line where you don't want

00:42:13.550 --> 00:42:16.730
to cross in terms of what you learn.

00:42:16.995 --> 00:42:18.120
At that particular time,

00:42:18.795 --> 00:42:21.825
Kert Viele: And there's an argument
there to get a little wonky in terms of

00:42:21.825 --> 00:42:28.275
the keeping the ratio between the arms
and phase two and phase three equal.

00:42:28.575 --> 00:42:32.415
So even if you get to the end, even
if there are slight differences,

00:42:32.475 --> 00:42:34.275
the control and treatment I.

00:42:34.565 --> 00:42:38.675
There are no systematic differences
between arms, which is important

00:42:38.675 --> 00:42:40.595
for, uh, interpretability.

00:42:41.870 --> 00:42:44.090
Joe Marion: Yeah, and from a patient
perspective, that means you might go

00:42:44.090 --> 00:42:47.990
from randomizing like three to treatment,
to one, to placebo, to one to one.

00:42:48.455 --> 00:42:49.805
In the, in the, in the last part.

00:42:49.805 --> 00:42:51.635
And that's, you know, something to think

00:42:51.710 --> 00:42:55.700
Scott Berry: we've done some where
it's, it's one to one to one in the

00:42:55.700 --> 00:42:57.260
first part, and then it's two to one.

00:42:57.290 --> 00:43:02.120
So you maintain the placebo proportion
throughout, and the formulas in this

00:43:02.120 --> 00:43:03.860
paper will allow you to do that.

00:43:05.135 --> 00:43:05.465
Joe Marion: Mm-hmm.

00:43:06.430 --> 00:43:11.915
Oh, oh, and then the other thing I,
I'm curious, you know what, if you

00:43:11.915 --> 00:43:15.905
don't wanna really pre-specify the
dose selection criteria at the interim.

00:43:16.775 --> 00:43:20.615
But you wanna let like a really small
unblinded group within the company

00:43:20.615 --> 00:43:25.355
see, like, I don't know, a couple of
efficacy endpoints summarized by arm

00:43:25.355 --> 00:43:28.505
and maybe the safety data and make
that decision kind of manual mode.

00:43:28.505 --> 00:43:30.065
Is that, is that an option?

00:43:30.070 --> 00:43:30.995
Have, have we tried that?

00:43:31.125 --> 00:43:34.425
Kert Viele: I, I mean, I'm gonna,
I'm gonna think out loud here.

00:43:34.575 --> 00:43:38.385
Shouldn't that be allowed
from, at least the math.

00:43:38.385 --> 00:43:42.735
I mean, we're picking the
best arm, so picking anything

00:43:42.905 --> 00:43:43.680
Joe Marion: it's good with the math.

00:43:44.655 --> 00:43:45.760
Kert Viele: So yeah, so the math

00:43:45.860 --> 00:43:47.660
Scott Berry: So let's
separate two scenarios.

00:43:47.660 --> 00:43:53.300
There's scenarios where A-D-S-M-B picks
the dose and the trial sets up that

00:43:53.300 --> 00:43:54.950
we're gonna allow them to pick it.

00:43:55.280 --> 00:43:58.700
Then there's no concern about
unblinding and the math all works.

00:43:58.700 --> 00:44:01.520
You can pick any dose and the math works.

00:44:02.210 --> 00:44:07.850
I think the scenario is, does somebody
at the company know something and I.

00:44:08.290 --> 00:44:11.230
Uh, you know, there are no
written rules about this.

00:44:11.230 --> 00:44:12.730
That's, that's a bit of the challenge.

00:44:12.940 --> 00:44:16.360
There are scenarios where somebody
very high up in the company, A CFO,

00:44:17.020 --> 00:44:22.930
A CMO, who doesn't talk to the sites,
is not involved in any way, executive

00:44:22.930 --> 00:44:24.730
level that's involved in that.

00:44:24.820 --> 00:44:27.100
And I I have seen that.

00:44:27.490 --> 00:44:30.550
So there are ways there can
be some level of unblinding.

00:44:31.010 --> 00:44:34.730
Uh, uh, to the company and all of
that, but you want to really keep it

00:44:34.730 --> 00:44:39.680
separate from sites involvement with
anybody who's talking to patients to, to

00:44:39.680 --> 00:44:46.010
really make sure that this is, um, you
know, it's hard to recover if somebody

00:44:46.010 --> 00:44:47.900
says, do you have operational bias?

00:44:47.930 --> 00:44:48.380
You can't

00:44:48.380 --> 00:44:48.620
show.

00:44:48.620 --> 00:44:49.190
You didn't.

00:44:50.045 --> 00:44:52.895
Kert Viele: so you may, you're still
double blinded, but maybe not triple

00:44:52.895 --> 00:44:54.665
or quadruple or quint blinded.

00:44:56.825 --> 00:45:00.035
Lindsay Berry: Yeah, and the math
in that situation works out for

00:45:00.035 --> 00:45:03.335
the type one error, you know,
because it's conservative, I guess.

00:45:03.335 --> 00:45:08.345
But you can't really simulate
that decision for power.

00:45:08.345 --> 00:45:11.840
You can't say, here's how accurate
our dose selection is if you don't

00:45:11.840 --> 00:45:14.015
have the algorithm pre-specified.

00:45:14.015 --> 00:45:17.225
So that maybe is a
little more challenging.

00:45:18.110 --> 00:45:18.770
Joe Marion: That's a good point.

00:45:20.000 --> 00:45:24.350
Scott Berry: All right, so thank
you all for joining here in this

00:45:24.350 --> 00:45:26.720
strange place called In the Interim.

00:45:27.140 --> 00:45:29.060
For now, thank you all for joining.

00:45:30.020 --> 00:45:30.470
Lindsay Berry: Thanks.

00:45:30.545 --> 00:45:30.995
Kert Viele: Scott.

00:45:31.055 --> 00:45:32.075
Joe Marion: Thanks for having us on Scott.

00:45:33.845 --> 00:45:33.965
Scott Berry: I.