Reflecting on EMA Reflections on Platform Trials

In this episode of "In the Interim…", Dr. Scott Berry and Dr. Kert Viele, Director of Research at Berry Consultants, review the European Medicines Agency's draft reflection paper on methodological aspects in confirmatory platform trials. They walk through the paper's definition of a platform trial and explain why the shared control arm is the source of both the efficiency and the complexity of these designs. Scott and Kert discuss the paper's preference for constant allocation ratios between each arm and control, its position that response adaptive randomization is not supported, two-stage randomization with multiple placebos, and the strong preference given to concurrent controls. They make the case that non-concurrent controls, adjusted for time, are high-quality controls, and welcome the paper's view that they are preferable to external controls. The episode closes with the risk of multiple false positive decisions from a common control, disjunctive and conjunctive power, and when multiplicity adjustment is warranted, with overall praise for a clear and well-constructed paper.

Key Highlights

• The shared control arm as the source of both efficiency and complexity in platform trials.
• Constant allocation ratios, and the paper's hard no on response adaptive randomization.
• Two-stage randomization, multiple placebos, and testing whether placebo groups can be pooled.
• A strong preference for concurrent controls, and the case for time-adjusted non-concurrent controls.
• Internal non-concurrent controls preferred over external controls.
• Multiple false positive decisions, and marginal, disjunctive, and conjunctive power.
• When multiplicity adjustment applies across arms, doses, and related families of hypotheses.

For more, visit us at https://www.berryconsultants.com/

Creators and Guests

Host

Scott Berry

President and a Senior Statistical Scientist at Berry Consultants, LLC

What is In the Interim...?

A podcast on statistical science and clinical trials.

Explore the intricacies of Bayesian statistics and adaptive clinical trials. Uncover methods that push beyond conventional paradigms, ushering in data-driven insights that enhance trial outcomes while ensuring safety and efficacy. Join us as we dive into complex medical challenges and regulatory landscapes, offering innovative solutions tailored for pharma pioneers. Featuring expertise from industry leaders, each episode is crafted to provide clarity, foster debate, and challenge mainstream perspectives, ensuring you remain at the forefront of clinical trial excellence.

Judith: Welcome to Berry's In the
Interim podcast, where we explore the

cutting edge of innovative clinical
trial design for the pharmaceutical and

medical industries, and so much more.

Let's dive in.

Scott: All right.

Welcome everybody back to In the Interim.

I'm your host, Scott Berry, and I have
my frequent co-host joining me today, the

director of research at Berry, Kurt Viehe.

Welcome back, Kurt

Kert Viele: Hello, good to see you

Scott: So today we are gonna reflect.

We're gonna reflect on
reflections perhaps.

Um, we, today our topic is EMA put
out just this week a, what they call

a, which is kind of weird, uh, a,
a reflection paper on methodologic

aspects in confirmatory platform trials.

We here at Berry love platform trials.

We're involved in many platform trials.

We've taken a number of these
to EMA, and so we heard this

reflection paper was coming out.

Uh, I was a little apprehensive.

Um, ICH E20, I, I…

Every time I reread it, I take it as more
and more negative and pessimistic, and

I, I suspect EMA had a large part to do
with that, so I was worried about this.

So Kurt, what, what do you
take on this reflection paper?

What's your, what's your first impression?

Kert Viele: So let's-- So
I, I'm, I'm certainly…

This is odd.

Aren't I meant to be the pessimistic one?

I'm gonna be the optimistic one today.

Scott: Yep

Kert Viele: so I, I think
this is l- a step forward.

Uh, as you said, you know, we certainly,
platform trials are near and dear to us.

We tried to count how many we've done
recently, and that was hard to do.

Um, but I think that this is,
we're certainly gonna have…

You know, there are places that
we do things that are actively

discouraged in the guidance or
in the, the reflection paper.

Um, so you know, we're gonna have a few
philosophical differences, but I think

on the whole, this is a step forward.

I think that one of the key things here
is this has been kind of a Wild West area.

You know, this is-- These didn't
exist in 2010, and now we've got

a hundred to a couple hundred of
these things around the world.

And so we haven't seen these kind
of guidance reflection documents.

The FDA now has one.

EMA is now putting out theirs.

And I think it's good to see there's a
lot of consistency here, and that will

help, uh, us, and it will help everybody.

Scott: Okay, let's, let's, uh…

By, by the way, my first impression was
and, and it's kinda like you see a movie.

If everybody tells you this is the
best movie you've ever seen, it's, it's

hard to meet that sort of standard.

So I came in a little bit worried.

So I came away with a
positive reaction to it.

But, but, but agree with it, I, I
do have some quibbles with this.

So first

Kert Viele: has anybody ever told
you this is the best guidance

document you've ever seen and

Scott: Uh, uh, interestingly, I first read
the Bayes guidance and I didn't, I wasn't

overly, "Oh my God, this is amazing."

And then everybody said that,
and I reread it and it was, uh,

this, this is pretty amazing.

Kert Viele: Okay

Scott: that, uh, by the way, the movie
that did that to me was Princess Bride.

Have you ever seen Princess Bride?

Yeah, yeah.

The first time I watched it, I
thought, "Oh, this isn't very good."

And then you watch it multiple
times, it's very good.

But, but we're, we're s- you're
getting me off topic, Kurt.

Kert Viele: at that

Scott: Yeah.

Okay, so let, let's set up a
little bit of the ground rules.

It, it talks about, at high level, this
is confirmatory trials for drug approval,

and that, that's specifically, uh, I
think they stay away from phase two

trials, purely learning trials in this.

They do…

I, I think they do a nice job
defining a platform trial.

There are many definitions
out there of platform trials,

basket trials, master protocols.

So the four bullet points they lay out
of, uh, what is a platform trial, and I,

I remember now, I'm trying to make sure my
wife Tammy can follow along as, uh, and I

think she knows what a platform trial is.

But, um, the first bullet is that
there are multiple treatments being

investigated within one trial.

Uh, I think that's pretty, pretty
standard, pretty good definition,

uh, a necessary thing of that.

Ran- uh, this is a really nice thing
that they say, randomiz- randomization

is used for treatment allocation.

There are umbrella trials perhaps where,
uh, patients can decide where they go.

Uh, it's entirely, uh, uh,
uh, investigator driven, so

randomization allocates to treatment.

There are multiple treatments.

They, they pretty much focus on
trials that have a control arm,

and they say that, that the arms
are being compared to a control.

You could imagine a platform trial,
a king-of-the-hill trial where

there are seven therapies and
we just wanna find the best and

there's no such thing as a control.

Uh, they're staying away
from such things as that.

And arms can be added or stopped
during the course of this.

So think of this largely as a continuing
perpetual process with arms that come

and go during the course of the trial.

Is, are there four definitions
of a platform trial

Kert Viele: Yeah, and I think one
kind of-- A-as people go through

this, one of the things to think
about in detail is that control arm is

really where all of the controversy,
complexity, wherever anything is.

On the one hand, we gain efficiency
because we're sharing the controls.

So we're instead of having, you know,
50 Alzheimer's trials, they all run

1000 control patients, we're going,
"Look, let's put these together.

We'll have fewer controls."

On the other hand, of course,
because they're sharing controls,

it depends on how we do that.

Can we pull the controls together?

Do we have to worry about as the
arms enter and leave, what do we do?

And we're gonna get into
that as we, we keep going.

But the con- everything
is about the control arm.

Scott: Yep, yep.

Okay, so, uh, I, I have a pet peeve, Kurt.

Kert Viele: wow

Scott: Yes.

Uh, Kurt usually has the
pet peeve of the day.

So I think platform trials are
just this incredible innovation.

You laid it out.

It has massive potential improvements.

Um, and so the fir- very first
part of the document, and this

is feels like an EMA way of doing
things, the purpose and scope.

So I, I'll read you from this, um,
I'm on line 40 of the document.

"Platform trials have become more common
over recent years and are likely to

play an increasing important role in
marketing authorization applications.

Design elements which frequently occur
in platform trials, such as adaptive

designs and common controls, increase
the complexity in planning, conduct,

and reporting of clinical trials.

This makes dedicated guise- guidance
necessary to enable adequate planning

and assessment of platform trials."

So it's, uh, it's, it's
not platform trials…

Could you have said something positive?

That this is a…

We're, we're excited about platform
trials, this is a good thing.

No, the, the, this guidance is necessary.

You all seem to be doing it.

We're gonna have to make authoring…

It just, it's a little Eeyore.

Uh, so that, that's my
pet peeve of the day.

The first start I was looking for a jolt.

Let, let's get people excited about
platform trials, and I got a little Eeyore

Kert Viele: Well, you'll need
to read the Bates guidance again

Scott: Yeah.

Yes, yes, yes.

Okay.

Okay, so it starts off, uh, and,
and you, you're welcome to go back.

I, I…

My first part I thought that it
becomes interesting, uh, o- is

where they start talking about
allocation ratios and probabilities

Kert Viele: Yeah, and I think this is,
um, a-and we should start with I, I think

we agree with most of this guidance.

We certainly agree with it when,
when a lot of the statements are made

on this is potentially a problem,
you need to pay attention to this.

We almost universally agree with
this, I think throughout maybe

a couple places we're gonna…

But, you know, we may disagree on
what's a easily surmountable problem

versus what's a horrible obstacle.

So the first thing's on
these allocation ratios.

One of the things to keep in mind is
you're comparing an arm to a control.

So if I enroll, say, two to one early in
the trial, I got two treatments for every

control, and later on I allocate three to
one or four to one, what's gonna happen

is early in the trial when I'm two to
one, got a certain number of treatments.

Later, if I go three to one, I now have
more treatment patients in the second

half of the trial as opposed to the first.

if we look at the treatment arm
and we go, "Hey, you know, when

were these patients enrolled?"

And we look at the control arm and go,
"When were these patients enrolled?"

They might be different.

So that, that's with potential bias.

So if you keep everything constant,
you always do two to one, always do

one to one, always do three to one.

Doesn't matter what you do,
but if you keep it constant,

you avoid all of these biases.

So that's kind of a base case, and then
we're gonna talk later about ways, times

we might wanna change the base case.

Scott: Okay, so they set up and they--
the-there's, there are multiple places.

They make a big deal in my, in my
reading of this that they want one

of the experimental arms, so you're
enrolling A, B, and C and a control.

They want the ratio of A to
control to stay the same.

If arms come and go,
it could stay the same.

So they bring up one-to-one
to one-to-one randomization.

So if a patient comes in,
they're randomized equally

among A, B, C, and control.

And if an arm goes away, it
becomes one-to-one to one.

So A to control is always the same.

I feel like that's what they want
people to do so that when they look

at A to control, that rate never
varied during the course of the trial

Kert Viele: Yeah, and the reason they
want that is it generates comparability

between the treatment and control
arms, and it does it directly.

There's no need for adjustments.

So it's the cleanest way to do
it in terms of the inference.

Not the only way to do it, but it's the

Scott: No.

I think they're picking a
little fight here with FDA.

Maybe you think otherwise.

Because the FDA guide said weird
specificity about doing one to one

to one to the square root of K, where
that's how many arms are in the trial.

So if you have four arms, it's one
to one to one to one to two, and if

you have three arms, it's one to one
to one to the square root of three.

Uh, and that would mean A to
control varies as arm, other arms

come and go in the trial, and
EMA seems to really not like that

Kert Viele: So I take it, and actually
it was really interesting in the, in the

master protocol and the FDA guidance,
it talks about that expressly and says,

"Oh, just model b- model it by time,"
which is what we would naturally do.

Um, and here it discourages it.

We've done platforms where
the number of arms change, and

we may do one-to-one-to-two,
one-to-one-to-one-to-one-to-two,

and we could keep that constant.

So, you know, I consider this-- I don't
necessarily see this as a big fight.

I'm not willing to…

I'm not willing to lose a lot of
sleep over that particular hill.

Scott: Yeah.

And, and the, the catch-22 in a little
bit of this is if you keep an arm to

the control always the same, you can't
keep the rate of placebo always the same

Kert Viele: Yeah.

And that there's a lot of things that
you can't-- A lot of people, what they

do is the primary analysis is the, it's
comparing the treatment arm to the pooled

controls, all the controls in the trial.

And there's often a sensitivity, which
is every treatment compared to its

matched set of controls, its specific
set that's most closely related to it.

Um, we'll kind of get into that a little

Scott: Yeah.

Right, right

Kert Viele: keep those
constant at the same time.

You have to

Scott: Right.

Kert Viele: here

Scott: Right.

So I think, I think interestingly,
all these randomization

things, you can model this.

They, they refer to it as
difficult or impossible.

It's incredibly straightforward to do time
adjustments for an arm, uh, within that.

Let, let's ignore
non-concurrent controls as that.

So this feels like not a complication
in trials, but they, they would

really prefer straightforward in
their marketing authorization.

That, that, that's clear

Kert Viele: Yes.

No, I agree with that.

Um, and to get back to your pet peeve
on complexity, I think when they talk

about preferring this, it's worth
saying explicitly what the assumptions

are that are being introduced here.

They're usually about, you know,
do you actually have a constant

treatment effect, for example?

This isn't-- It's not mystical.

We know what this is.

It's worth discussing, but we should
say it rather than leaving it as a,

you know, something might happen.

Scott: Okay.

A- and they do talk about
modeling, uh, time periods.

Kert Viele: Yep

Scott: end of this section, they
have their first, and maybe perhaps

only hard no, uh, in the document,
and that's kinda the first thing

I look at is, is there a no?

Because discouraged means I can
convince them it's appropriate in

this scenario if it's appropriate.

I mean, we don't do things
unless there's need and purpose

and the science matches it.

So I feel like we can do that.

But here, there's a hard no

Kert Viele: So we see Scott's
management style right here.

You know, there, there's the people who
manage and it's sort of like, "Well, I'm

only gonna do what they tell me to do."

And then there's the people who go, "Well,
they didn't tell me I couldn't do it."

You're firmly in that bucket.

Yep,

So

Scott: yep.

And so what, what's the hard,
what, what is the hard no?

Kert Viele: So the hard no in there
is on response adaptive randomization,

which is essentially you're explicitly
changing those randomization ratios,

and, uh, and many times you're doing it
often, so they go up and down and so on.

Um, now this is probably where
you and I may actually have

our own internal disagreement.

I don't necessarily right
now consider that a big loss.

Um, I think a lot of arm-dropping
designs do as well as RAR, and so

the ones at least that I've been
designing, I don't have RAR in them.

Um, I think it's-- might be useful
for us to think about, you know,

where we need RAR, um, and you
know, where is it gonna be valuable.

But I personally, I'm okay
with the no RAR the time being.

Scott: Yeah, and I, I think because
they're focusing so heavily on

Phase III authorization, it's,
it's pretty rare that RA has done.

Now, we have several Phase III trials.

Uh, GBM AGILE does response
adaptive randomization.

Uh, uh, Precision-PROMISE does it.

Um, and so I-- the idea of a hard no,
I think, is just, I, it, yeah, I think

it's a little closed, uh, in this.

But I, I agree with you that I,
I, I don't think it's a huge loss.

It, it's reasonably unusual for it.

But, but part of it is, uh, now
I'm, now I'm gonna pick a fight

with you on something you just said.

Yes, yes.

So their, their justification of it
is that changing allocation ratios

over time for an arm, um, relative
to the control is why they don't

like response adaptive randomization.

Fair?

Kert Viele: Yeah, I think that's fair

Scott: So they bring up the
possibility that you could get

enrolled to a particular drug which
has two doses and or a control.

If you arm drop one of those two
doses during the course of the trial,

you may be altering-- you will be
altering the rate of an arm to control.

Now, it's not response adaptive that
frequently changes to, to go after a

better arm that A is doing better than B,
but it brings about the same issue, which

I don't think would be an issue for them

Kert Viele: I think, uh, first off,
I'd like to note that if we were

getting in a bar fight, you could
just call me ugly or insult something.

It's really, really quick.

You wanna pick a fight as a
statistician, there's like a

four-minute preamble of all this

Scott: right.

Kert Viele: The,

Scott: Yeah

Kert Viele: But anyway.

No, I, I think that's right.

Um, I think there are cases
where it would be valuable to try

to accelerate pers- some arms.

Um, I'm not, probably not
quite answering your question.

I'm deflecting it a little bit.

I think RAR is useful when you, you speed
up more promising arms and slow down less

promising arms, you get the answer faster.

So I can certainly see some cases
where, hey, if you gave me the

design and said, "I can show this
does a lot better," I would at least

hope the EMA would listen to that.

I just think right now those are…

If you told me I can't do RAR, I
think I can do almost as well with

something that will fit in what
they prefer, which I'm okay with

Scott: And, and in a confirmatory trial,
the place where I think it would have

the biggest role in a platform trial
is when you have multiple subgroups.

Uh, so you have four subgroups and drug
A is doing great in subgroup one and

drug B is doing great in subgroup two.

You'd like to be able to
personalize that in response.

The EMA shuts that down, that this
is about one patient population.

Uh, and so this is not
platforms with baskets.

Uh, and so maybe if they do a reflection
on platform trials when there are subgroup

effects, they'll bring RAR back in.

Can I, can I hope?

Kert Viele: You can hope, and actually
it's kind of ironic 'cause I-SPY

2, one of the first platforms, was
almost one of the most complex ones

'cause it was phase two, but it did

Scott: Yep.

Yep.

Yep.

Yep.

And, and by the way, GBM Agile has
that characteristic that it has

subgroups, and that, that's a huge
benefit of platform trials to be able

to enrich, which is, which is RAR.

Okay.

So I got out of my, my hard no
without being, uh, too, too critical.

And I, I wasn't surprised.

Okay.

So by the way, it does, it does talk
about multi-stage randomization.

So first of all, explain
multi-stage randomization

compared to a single randomization

Kert Viele: All right, so suppose I have
four arms, A, B, C, and D, and a control.

What I could do, for example,
is I could just take patients.

I've got five randomization probabilities,
control in the four arms, and I just

basically roll a-- there's no such
thing as a five-sided die, but on my

computer, I roll a five-sided die.

So I can just do that directly.

The problem that comes up is you
have to, uh, you have to consent

patients for all four of those
arms, which could be complicated.

I gotta talk about A, talk about B, talk
about C, talk about D, explain all the

side effects, how we're randomizing.

It's complicated in order
to do that for a patient.

So I've just been in the
hospital with relatives.

You know, it's a hard time to be,
"Hey, let me explain a whole bunch

to you at this particular moment."

So in order to handle that, one way that
platforms work is they, they screen-- they

consent you and screen you in two phases.

In the first one, they say, "Come
into the platform, we'll do all our

baseline labs and everything else,
and we will randomize you to what we

don't really have a name for yet,"
but I call it cohort, and other people

don't like that they call it cohort.

it basically says, "Okay,
you're in cohort B.

You're either gonna get active B or you're
gonna get the control version of B."

And so at that point, you only
have to explain B to the patient.

You don't have to explain
all the other arms.

If arms come in and out, it
doesn't affect it anymore.

You just have to talk about B.

so then they do the randomization
there in that second stage.

This also facilitates blinding.

If A, B, and C, you know, some of them
are oral, some of them are injectables,

the control version can be matched
to that as well, is another aspect.

The good part of that is
it's operationally easy.

The bad part of that is now you have to
worry about, can I pool those controls?

The patients that are in control
A, control B, control C, control

D, can I put them together or not?

Is, is where we're gonna go next.

Scott: Yep.

Yep.

And, and, um, uh, that multi-stage could
be done even potentially when there's a

sh- they, they, they do differentiate a
shared control from multiple placebos.

So, uh, I was a little bit worried
when-- So, a- as Kurt described this

multi-stage randomization, it's a way in
which patients can be ran- you can have

a placebo for each of A, B, C, and D,
and a patient is randomized to cohort A,

and they get active A or placebo of A.

You're going to have patients that get A's
placebos, B's placebos, or C's placebos.

No trial I've ever heard of, platform
trial, tries to triple blind that you

don't know if you're an A, B, C, or D.

It's actually, it doesn't
work if arms come and go.

You can't do that.

Uh, and it, it would be incredibly
horrendous that a patient takes

three modes of administration of
a placebo, and it's unnecessary.

So that's a, that two-stage randomization
is very, uh, straightforward in that

particular situation and easier for
a patient, which is a great thing

Kert Viele: So what people
worry about there is the kind

of things that can go wrong.

Suppose that-- So we just did a PTSD
trial uh, I won't get into which drugs.

I'm not sure I can get into which drugs.

But let's just say one of the drugs
we were pretty sure people were

gonna be excited to be get, and one
of the drugs we-- people were pretty

sure where they were gonna be like,
"Ugh, I could do that anywhere."

Um, so we were really worried people
would enter the platform and they get

randomized to the unexciting drug,
and they just go, "Okay, I'm not

gonna consent at the second stage.

I didn't get what I wanted."

Whereas the exciting drug,
everybody would enroll.

And you can imagine that generates
different populations, and that

would be a threat to pool ability.

other cases, the drugs may be
reasonably the same excitement level.

The excitement level
might not affect results.

This is something you need to
discuss, but it's not a deal breaker.

So

Scott: Yep.

There's, there's other
trials that we worry about.

One, one might be one pill a day, and the
other is, uh, you know, uh, an infusion,

uh, once a week during the time, and it's
a, a, a, you know, four-hour infusion

that people are not gonna wanna do that.

And so when they get that mode of
administration, they're out, and

you get differential dropout in
there, which, which clearly could,

could affect, uh, pool ability

Kert Viele: You could have
safety labs, for example.

One drug you have to do extra
safety testing for some reason.

That may or may not…

I mean, if they're already getting
blood draws and it's another thing

in there, yeah, you're gonna do
it for only one group, but it

probably doesn't affect the results.

So there's a whole continuum from,
oh yeah, this is gonna be a disaster,

you really have to worry about this,
to okay, this is a minor problem

Scott: Right.

And, and the steering committee's gonna
worry about it if it's the case that

it's a, uh, y- you know, are we doing
gene therapy randomization relative to

a, a, you know, a small molecule that
people's very perception about what

they want may be very, very different.

You would very much worry about that
i- in those trials, rightfully so.

Okay.

Uh, I thought they might make a bigger
deal about multiple types of placebos.

It was one of my, my worries coming in,
and I think they mention it, discuss it.

Um, one of the, one of the amazing
things about this is you will have

randomized placebos, diff- comparable
pla- A patient will be eligible for

three different placebos, and you will
have data on a randomized comparison

of A0 B0 and C0 and you'll be able to
empirically investigate are they different

Kert Viele: Yeah, it's, it's
not an untestable assumption.

It's a very testable assumption.

It may not have the highest power
depending on the sample size,

but that's a choice you can make.

If you're worried about the
assumption, enroll more controls

and make sure it is testable

Scott: Yep.

Yep.

Uh, by the way, I don't think we've
ever seen one where that has mattered.

Kert Viele: No, and we have posters.

The, the HEALEY trial has
released a poster on this

that, uh, things were equal.

And I fully agree this may depend on the
edu- on the indication, not education.

But, you know, on ISpy-2, for example,
I don't think it ever mattered.

Um, the, um…

Anyway, it's-- it often doesn't

Scott: Yeah, yeah.

You could imagine a trial in pain.

PTSD is one that you would
potentially worry about.

Kert Viele: Ja

Scott: Okay, so I think we
got out of there unscathed.

We got-- We didn't get a
hard no, uh, in all of that.

Uh, okay to move on to
concurrent versus non-concurrent

Kert Viele: Yeah.

So let's talk about-- So the other
thing that comes up, this again all

related to the control arm, when
we're pooling these controls, the

question is do we use all of them?

And usually we put some
restrictions on these.

We put restrictions on, uh, what's called
concurrent controls, which means controls

that were enrolled at the same time as the
arm that you're doing the analysis for.

So whenever arm B was enrolling,
you only look at the controls,

pooled controls that were enrolled
at the same time B was there.

That's to avoid time biases.

Usually, this is actually becomes--
It's be- I actually don't like the

fact we still call them concurrent
because we've added things.

Usually, we restrict for eligibility.

If patients aren't eligible
for an arm, they go away.

They're not in the pooled
controls for that arm.

Um, and that could, you know, certain
sites may not enroll all arms.

A lot of times we would
eliminate those sites.

We may not, so this is, you know,
the, the REMAP, the Tamiflu issue.

There was a long discussion
about what to do there.

Um, but all of these things, the basic
idea is this causal inference idea

of you only include the patients who
could have gotten both of the arms.

And so all of this is aimed at how do we
make sure we're only looking at those.

Scott: So a- and to set this up a
little bit, you could imagine the trial

starts with A and B and controls, and
they're, they start enrolling, uh,

and six months later, C comes in the
trial, and it's inserted, uh, uh, in

real time, and all of a sudden, six
months into the trial, patients can

be randomized to A, B, C, or control.

Now, when C is going to be
analyzed, for six months, controls

were enrolled before C was there.

Those first six months of controls
would be considered non-concurrent

controls, where after the six months,
they would be concurrent controls to C.

Those initial six months were
concurrent for A and B, so it's,

it's relative to the particular arm.

So when C reads out, the question is,
do we use the controls from the first

six months, is, is the discussion.

Their…

and, and their phrase on this, uh,
and th- by the way, this is a topic

that Kurt and I did a whole separate
podcast on, I think two podcasts on the

Bayesian time machine and all of this.

So if you're interested in this
topic, uh, go back and listen to this.

By the way, I, I think we have
Janet-- we have convinced Janet

Wittes based on that, that adjusting
for time is the right thing to do.

Kert Viele: Hi, Janet

Scott: yes.

So thank you, Janet.

That's wonderful.

Uh, by the way, I'll see Janet in three
days at a conference, so, uh, she won't

have heard this, uh, in, in the time.

Okay

Kert Viele: I hope at the point
you will have told her hi for me.

Scott: Yes.

Yep, yep.

Okay, so the phrase in here
says, "A strong preference is

given to concurrent controls."

So this goes back to Kurt's comment.

A strong preference is given
to concurrent controls.

That's a license to me.

Uh, you know, this is, this is great.

That's not a hard no, um, uh, in it.

And, and I think they even
give this phrase, uh, in unique

situations it might be appropriate
to use non-concurrent controls.

I'm gonna go to them with lots of
unique situations, I think, uh, in it.

Okay.

Kert Viele: And we, we should
emphasize there are lots of our trials

that do use non-- do use concurrent
controls, and we fully agree that

concurrent controls are much more
valuable than the non-concurrent.

They should be weighted more.

I think a confusion here is people
think we're gonna pool everything.

That never happens.

When we're doing this analysis,
it's gonna get down-weighted

the farther back in time you go.

The question is just do you eliminate
them or do you use them to some degree?

Scott: And I give, I give huge credit to,
um, this document for a couple things.

First of all, we're in a world now
where everybody's talking about

real-world evidence, external data,
digital twin, virtual patient.

I, I mean, we are being inundated
now with, uh, external controls.

They actually have a phrase that…

A- and by the way, imagine in,
and they say this in the document,

and they say, "If you're thinking
about external controls, we'd much

rather have you use non-concurrent
controls in the same platform.

They're much better."

Kert Viele: which we completely agree with

Scott: Oh, it's, it's, it's…

They're the greatest disconnected
controls or something.

It's the same protocol, same inclusion,
exclusion criteria, criteria.

They've all consented.

They were all blinded, or they
were all part of the randomization.

The same endpoints.

They came three months earlier.

Uh, they're the greatest controls
you're ever going to get in a situation,

but it's even more better than that.

It's, it's not like these, the, in
that six months you have C and you have

the concurrent controls, and then you
have the, the, the ones previously,

and you don't know if it's different.

You have A and B that were enrolling
at the same time in those six

months, and they were enrolling after
that six months concurrent to C.

You have a connected network to
identify, did the controls behave

differently in that first six
months to the next six months?

It's a simple adjustment model,
which by the way, Kurt has written

quite a bit about in that scenario.

They mentioned that

Kert Viele: Well, it's, it's
just as you said, it's a s-

it's a different assumption.

If you'd use the non-concurrent
controls, you use all the arms, you need

treatment effects to be constant in time.

If you use external controls, you--
most of the time, you need the

treatment effect to be the-- You
need the treatment itself to be…

Or sorry, the control's parameters
to be the same over time.

It's a stronger assumption
in a lot of ways.

So the non-concurrent are better

Scott: Yep.

And so I think I, I, I'm amazed
continually by how much people talk

about real world evidence and they
talk about this, but oh my God,

don't do a non-concurrent control.

I mean, it's just, it's so striking to me.

I think it is the least risky
thing and incredibly valuable.

Think, think about an ALS trial that not
using them means we're gonna continue

to enroll a bunch of, of, of patients on
placebo in a deadly disease rather than

doing a simple model that adjusts for
time, which means we might be able to

enroll more arms in our trial because we
don't need to enroll so many controls.

Would you rather use non-concurrent
controls and have five arms in a trial or

three arms and use concurrent controls?

I sure hope you'd rather see
five take more shots on goal.

So yes, I get excited about this.

I, I, a- and, you know, it, it bothers
me that this bothers people so much

'cause, oh my goodness, this is not hard

Kert Viele: It is just adding a covariate.

We, our time machine model, it does
some smoothing, but you can just put,

just bin by time and add a covariate.

It is adding a covariate to the model.

We have this technology

Scott: Yes, we, we do
have that technology.

Uh, that was, that was actually
a, a laugh-out-loud moment.

That was good, Kurt.

L- LOL.

Yeah, yeah.

Okay.

Okay.

So, uh, uh, it says strong preference,
it doesn't say no, so I'm good.

I think we can make the unique argument.

I do worry that, uh,
it's not read the same…

Scott, other people don't read it
the same as Scott, but yeah, okay.

Um,

Kert Viele: The real question is,
did they write it the same as Scott?

Scott: Yeah.

Yeah.

Oh, I give them huge credit for
mentioning you can do this time

adjustment, and these controls are
incredibly high-quality controls.

But boy, you make our life easier
if you use concurrent controls is

the, is the sort of way I read this

Kert Viele: I said, we agree
on everything on what's better

than-- X is better than Y.

We have disagreements on
how much better X is than

Scott: Yep.

Or the implications that
have to the global, uh, you

know, treatment of a disease.

Um, you know, the- these are
not free by, by any s- yeah.

Okay.

Okay, um, good to move on?

Kert Viele: Yeah, please

Scott: Okay.

The next section, uh,
blinding, uh, in this.

I thought this was pretty good.

They talk about it being burdensome
to read, uh, that, you know,

that such a proce- to triple
blind would be too burdensome.

They kinda dismiss it, that that's
really good that they're not pushing that

Kert Viele: I actually thought this whole
section was better than the FDA's master

protocol guidance in terms of I thought…

And part of that's because the FDA
went first, but I think you're seeing

the evolution of this where people
are realizing the operational issues,

and I think this was handled very well

Scott: Okay, let's go from there
to, I, I don't know, I, I, I

almost LOL'd during reading this.

Uh, the n- the, the next section
maybe it's, uh, is risk of

multiple false positive decisions.

This is something…

I was shocked by the amount
of space they spent on this.

I, uh, really weird.

Take, te- d- explain what this
is, Kurt, and were they justified?

Kert Viele: So I think this is an issue.

I think the-- Okay, so let
me explain what the issue is.

Definitely a thing.

Um, so what's gonna happen is you've
got this one control arm, and you're

comparing, you know, three or four
or five different treatments to it.

If you were to get a spurious
result in the control arm,

it's randomly low, for example.

So now the control's down.

Suppose that everything's
a null, nothing works.

Well, the control's down, so that
makes it a little easier to reject A,

a little easier to reject B, C, and D.

And so this random low in the control
arm could make you reject everything.

You could make a lot of
false po- false positives.

And it works the other way, too.

They don't talk about it as much,
but it could be a random high.

You lower power in every arm.

the worry is that is it bad that we
have an increased risk of multiple

Type I errors at the same time?

One of the things that's hard here
is there's a compensation to that.

While we have more risk of a lot of
Type I errors, and I should be careful.

When we're saying more risk, we're talking
about this is well under one percent.

Scott: Yeah, right

Kert Viele: talking
about a ten percent risk.

We're talking there's some
decimal places attached to this.

On the other hand, of course, we have a
lower chance of making any Type I error at

Scott: Hmm

Kert Viele: in those cases.

So, and I think you see
this in the document.

They talk about the trade-off here,
and it's not clear that they can--

they, they don't have a very good
description of how to balance that.

It's sort of this is an issue, and then
it, the, it moves on to something else.

So this exists, but I don't know
that it's necessarily a problem,

so that we're, that we're here

Scott: And, and that was my take.

I, I don't think it's a problem.

It's incredibly unlikely, and, uh,
nobody really does anything about it.

Now, you could enroll a bigger control.

And why I was laughing is, imagine in the
middle of your trial, year two to four,

that you've got this case of three arms
and a, and a concurrent control in that.

And you have two years of prior controls
from the same trial with overlapping arms.

Boy, you reduce the probability of
multiple false positive ones by using

the non-concurrent controls, which
they said, "Ooh, we'd like you not

to do that," but boy, we're worried
about multiple type one errors

Kert Viele: By the way,
I do like enrolling.

They in many cases say
enroll more controls.

I am all in favor of that.

I'd actually be, and I, I realize
sponsors may not be on this, I'd be

okay if taking alpha down a little bit.

Suppose I onset a two and a
half percent type one error,

it's two point two five or two.

There's a lot of efficiency
in a platform trial.

I'd be willing to trade some of that for
robustness or various other things, so,

Scott: Yeah.

Kert Viele: substantial savings.

Scott: Yeah, I'm, I, I…

Yes, I, but I think you're giving
away something for a, a th-

Kert Viele: away.

Scott: a hypothetical concern.

Kert Viele: That's why

Scott: Okay.

Kert Viele: negotiate with anybody.

It's a good i- good idea.

Scott: Okay.

So I, I found it funny that they
spent that much time on that.

And when you simulate these trials…

By the way, they even talk in the
operating characteristics about

simulating and, and calculating these
things, and they, they create terms.

Uh, you know, we've, we've
simulated 50 platform trials.

We're, we're trying to count them
actually, but, um, uh, in that, and they

talk about disjunctive and conjunctive
power, and cal- uh, you know, at, what's

the probability you accept all of them,
you accept at least one of them, marginal

power, but also, uh, type one error.

How often do you make two type one
errors in a particular situation?

So they seem very focused on this

Kert Viele: Yeah, I think--
So what do we typically do?

Marginal power, the power for
each individual hypothesis.

We do this

Scott: Ja.

Kert Viele: time.

Scott: Ja.

Kert Viele: We're often interested in if
we run the platform, what's the time until

we find the first effective treatment?

That's been a big deal in ALS
and PTSD, hard to treat areas.

We want to know how quickly can we
get good treatments to patients.

Uh, I don't know that we've computed
these disjunctive conjunctive power ever

Scott: I, I, I just maybe for curiosity,
but I, I don't think we've presented them.

Yeah, yeah

Kert Viele: So it's certainly the
third thing on the list in the--

but marginal power is by far the
thing we focus on in most of ours

Scott: And I thought they
do a really nice job.

I mean, what we're not saying is, "Oh
my God, EMA says when a third arm comes

in, you need to adjust the second arm."

Uh, you know, absurdity.

So, so they recognize and they talk about
in pretty good detail what i- what are

the questions you're trying to answer.

Kert Viele: Ja

Scott: if you're trying to answer does A
work and separately does B work, that you

don't need to adjust one or the other one.

Um, largely they say that.

Um, and I, I thought
that that was positive.

They even talk about related
families in, in the platform

Kert Viele: Yeah, I mean, they
give the standard, the, the obvious

place to potentially adjust is
multiple doses of the same drug.

I think this gets complicated when
you have two drugs that have the

same mechanism of action that go in.

Anyway, but I think actually they
do a really nice job in this area

of recognizing this is gonna be
indication specific, and I, I

think that was all quite fair in
terms of the descriptions there

Scott: Yep.

Yep, I did too.

I, I, I thought they did a
nice job other than the, the no

on RIR, which I, I, I sort of
understand, uh, within that agreed.

All right.

Um, those, those were the sections I
highlighted and thought were interesting.

Did I miss any, Kurt?

Kert Viele: I think we, we pretty much
went through all of them, didn't we?

Scott: I think we did.

Kert Viele: So

Scott: we did.

There are, by the way, if
you're-- You, you can go to

the EMA website and get this.

There are 507 lines to it, so it's a,
you know, 15-page, 14-page document.

Kert Viele: Very accessible.

Scott: it a read.

Give it a read.

Yep.

All right.

Um, uh, exciting.

It's always good to get this.

Um, uh, I, I-- It is well done.

I think they, they, they did a nice job.

They explained things pretty clearly.

We, we pointed out a few things
we, we think are, uh, a little

awkward, if not conflicting.

But largely, I think they
did a really nice job.

So I, I, I commend them on them
and commend them for not, uh,

saying I couldn't do things

All right.

So thanks, Kurt, for reflecting today.

Uh, appreciate your reflecting, and
thanks everybody out there for listening.

Until next time, we'll
be here in the interim