Failure of the Proportional Odds is the Result

In this episode of "In the Interim…", Dr. Scott Berry and Dr. Elizabeth Lorenzi systematically examine the analytic pitfalls in recent acute ischemic stroke trials, especially the implications of violating the proportional odds assumption on the modified Rankin Scale. The discussion draws on the DISCOUNT, INSTANT, ESCAPE-MeVo, and ORIENTAL-MeVo trials, spotlighting frequent reactive shifts to proportional odds violations. Scott and Liz detail how such approaches obscure clinically relevant heterogeneity and react by creating analysis methods that obscure the clinical relevance of a violation of proportional odds. The episode underscores the necessity for trial designs that explicitly address heterogeneity of treatment effect. Listeners gain an unvarnished critique of prevailing reporting practices and an actionable vision for future stroke trial designs.

Key Highlights

• DISCOUNT trial’s interim analysis and futility stopping.
• Issues with endpoint dichotomization after proportional odds violations.
• Comparison across multiple recent stroke trials with inconsistent endpoint definitions.
• Obscuring of proportional odds violations, which may be the most important result of the trial.
• Adaptive strategies in STEP platform.

For more, visit us at https://www.berryconsultants.com/

Creators and Guests

Host

Scott Berry

President and a Senior Statistical Scientist at Berry Consultants, LLC

What is In the Interim...?

A podcast on statistical science and clinical trials.

Explore the intricacies of Bayesian statistics and adaptive clinical trials. Uncover methods that push beyond conventional paradigms, ushering in data-driven insights that enhance trial outcomes while ensuring safety and efficacy. Join us as we dive into complex medical challenges and regulatory landscapes, offering innovative solutions tailored for pharma pioneers. Featuring expertise from industry leaders, each episode is crafted to provide clarity, foster debate, and challenge mainstream perspectives, ensuring you remain at the forefront of clinical trial excellence.

Judith: Welcome to Berry's In the
Interim podcast, where we explore the

cutting edge of innovative clinical
trial design for the pharmaceutical and

medical industries, and so much more.

Let's dive in.

Scott: Welcome everybody
back to In the Interim.

I'm usually your host, Scott Berry.

Uh, I am Scott Berry by the
way, but I'm usually the host.

But I have, I have a, a deputized
host with me today, and I'll

explain that a little bit.

But Dr.

Elizabeth Lorenzi, uh, Liz, welcome
and, uh, thanks for, for trying to keep

me on the straight and narrow today

Liz Lorenzi: Yeah, of course.

Thank you for having me

Scott: So I, I asked Liz to join me
because I, I have a topic and sometimes

get a little passionate about it, and, uh,
uh, we'll come to this, and she's going to

keep me on the straight and narrow here,
um, and, and kind of play the role as I,

as I walk through a sequence of things,
and I'll explain a little bit what it is.

Uh, she's gonna make sure that somebody
in the audience actually is, uh, getting

a, a productive, uh, discussion today.

Okay.

um, how did this come about?

And, and, and by the way, I asked, I
asked Liz to join for this because we

have been working on the STEP platform
trial, and acute ischemic stroke, Liz

has a great deal of experience in acute
stroke, so all of this is very familiar.

And so, uh, she's, she's gonna be
able to put this in context for us all

Okay, so i-it's become something inside
Barry Consults of almost a weekly

occurrence, but are tons of trials
coming out in acute ischemic stroke.

And it seems like almost every week
they come out, and it's a bit of a, a

way that people can poke fun at Scott,
is to present the primary analysis

of these trials, because I, I, I
completely get triggered by dichotomous

analysis, uh, in stroke trials.

So I love looking at the results of
these trials, and we even have a website

where we post all of the results of them
and every way you could analyze this

modified rankin and what the result is.

So just the other day, Discount
trial came out, um, which is kind

of an interesting name of the trial.

Uh, the Discount randomized trial,
and I'm gonna walk through the results

of this, and it is a little bit of a
story, and hence, uh, Liz keeping me,

uh, on the straight and narrow here.

Okay, so the Discount trial is a trial,
and first I'll, I'll describe the patient.

So it's medium distal
vessel oc- occlusion.

this is a stroke, and this is a, a, a
clot, uh, of a vessel within the brain,

and the opposite of medium or distal,
I think they're small, but large vessel

occlusions, we'll come back to that,
have been enrolled and showed actually

tremendous benefit for the treatment
in this trial, which is endovascular

therapy, clot clearing device.

So they go in and they clear the clot, and
huge benefit in large vessel occlusion.

So the, the big question is,
is there benefit in the medium

distal vessel occlusions?

So they enroll, uh, there, there are
multiple classifications of these M2

distal, uh, MCA, A1, A2, A3, anterior
cerebral artery, all within this

classification of medium vessel.

within, uh, eight hours from onset
of symptoms, 24 hours since last

known well, which can be an important
modifier of benefit potentially.

And the patients have to be NIH
stroke scale, um, of five or greater.

And the higher, the more
severe the symptoms of this.

The-- and so these are at least
having some severity of symptoms

with medium vessel occlusion.

They enrolled them in the trial,
and the intervention is endovascular

therapy, so clot clearing versus
standard medical management.

Okay, so have I explained
the Discount trial, Liz?

Liz Lorenzi: Yes, I think so.

Um, I-- could you just clarify a little
bit on the last known well inclusion?

So you said it's within twenty-four hours?

Or what was the eight part too?

Yeah

Scott: so patients were admitted within
eight hours of onset of symptoms.

Liz Lorenzi: Okay

Scott: if they generate s-
or 24 hours last seen well.

So I think a, a wake-up stroke,
it's unclear when symptoms started.

So I think if it's 24 hours since
i- if they find somebody sitting in

a chair in front of a TV, when was
the last time they saw them well

because the person can't report

Liz Lorenzi: Okay.

Scott: I think

Liz Lorenzi: Okay, makes sense

Scott: Okay, the primary endpoint
in the trial is the modified Rankin

score, and, uh, lots of episodes, you
can go back to the discussion of this.

It's a seven-point ordinal scale,
and I think general agreement from,

uh, everybody in the stroke community
that this is a really nice measure

of the, the clinical status, the
neurological status of a patient.

And the scale is seven points from
zero is no symptoms, so the best

possible outcome is zero, neurological
status, to six, which is dead.

And it's just important grades from zero
to one, one to two, two to three, three

to four, four to five, five to six.

The little bit of the controversy
is five is severe disability, almost

vegetative, whether five and six a
lot of times are lumped together.

So you could think of as a six-point…

as perhaps a, a six-point
scale as opposed to seven.

Zero to six is seven outcomes.

Okay, and then the question is,
how do you analyze this scale?

So how did Discount analyze the scale?

They analyzed it where they lumped
in as a dichotomy zero to two

against three and above this trial.

So medium vessel occlusion, endovascular
therapy, zero to two is the, the, uh,

responder success against a failure.

Liz Lorenzi: And what do you, what
would you say zero to two captures?

Is that, you know,
small, small disability?

I guess it's goes up to slight disability

Scott: Yeah, sometimes this is
referred to as functional independence.

Liz Lorenzi: Okay

Scott: if you're two, zero, one, or two,
you pretty much function independently.

You have disability, you may have trouble,
you may need a cane kind of things.

Three is moderate disability, and so
you require some level of assistance.

Uh, so they refer to as
functional independence.

Uh, but then three to six there, you
know, there are four grades there.

Liz Lorenzi: Okay.

Scott: sometimes labeled as zero to two.

Liz Lorenzi: Got it

Scott: an, not an uncommon way
stroke trials are analyzed.

Okay.

So interestingly, I, I, I saw that
the trial-- happened in the trial,

interestingly, they had a group
sequential trial where they were

doing two interim analyses, and the
first one took place with a hundred

and sixty-three patients randomized.

And so they're, at this point, they're
doing a group sequential test, and the

trial ended up stopping for futility.

The Data and Safety Monitoring
Board recommended to stop the

trial for the following reasons: a
lower efficacy for the experimental

group, so it was doing worse.

Over, uh, they, they saw, uh,
i-intracranial hemorrhages bleeding in

it, and the conditional power was less
than ten percent of the trial being

successful at the maximum sample size.

So the DSMB stopped the trial.

Wasn't a predefined trigger, but
they stopped the trial at that point.

Liz Lorenzi: Oh, so that
futility rule with conditional

power, did they meet that?

Was that a formal rule

Scott: I think that was
the reason the DSMB gave

Liz Lorenzi: I see.

Scott: stopping it.

Liz Lorenzi: But it wasn't
a pre-specified trigger

Scott: Uh, my reading of it, no, it was

Liz Lorenzi: Okay.

Scott: Yep.

Liz Lorenzi: Interesting

Scott: They had, they had a pre-specified
rule that was not triggered, but they

stopped it anyway, and they gave, by the
way, the conditional power was less than

10%, as additional justification to it
was doing worse and there were more bleeds

Liz Lorenzi: Okay

Scott: So the, the result of this was
that it-- the, the final result when the

data were all read out is that it was
doing worse on that primary endpoint.

They saw a reduction of
about five pers- was it 7%?

Uh, six and e- 6.8%

absolute in the zero to twos.

I noticed, of course, that
interestingly, they had an improvement

on thrombectomy in zeros and ones.

So that difference was actually
a positive for the device.

But then when you add in the twos,
there were twenty-one percent

twos on the device, thirty-two
percent on medical management.

So the sum of zero, ones,
and twos was worse on device.

Zeros and ones was better.

But with the primary endpoint of zero,
one, and two, they stopped for futility

Now, is that kind of interesting?

And, uh, let me tie in also the…

This was in JAMA, so this discount
was a recent publication this week

in JAMA, and, um, the conclusion in
JAMA was thrombectomy did not increase

the rate of good clinical outcome

Okay, so put that in the back of your
mind is the way they summarize this.

It did not increase the rate
of good clinical outcome.

Interestingly, uh, thinking back
in this, you know, all these stroke

trials that come out and I, I read
the result, how do they analyze them?

How do they interpret the results of this?

very recent trial was the INSTENT
trial, and the INSTENT trial came out

in May of twenty twenty-six, a very
recent trial, where in that trial,

the, the primary endpoint was zero
to one, uh, dichotomous, and this is

a drug treatment after tenecteplase.

If the patient isn't doing great,
do you give an additional tirifuban?

Um, and so it's different than
endovascular therapy, but the

primary endpoint was modified Rankin
score, and they analyzed zeros and

ones and not zero, one, and two.

And in that one, they showed a
statistically significant improvement

on zeros and ones for the treatment.

And the conclusion in JAMA, same
journal, was that the adjunctive

intravenous tir-tirifuban increased
the likelihood of an excellent outcome.

different, but improved, improved
outcome in, in that trial.

Interestingly, if you analyze
zero, one, and two of INSTENT, it

is not statistically significant.

You know, so ha- if in Discount, had the
primary endpoint be zero, one, and two,

would the DSMB have stopped the trial?

Probably not

Liz Lorenzi: was it, a
different population in INSTANT?

Scott: Yes.

Yeah,

Liz Lorenzi: Okay

Scott: population.

So, uh, the, uh, I don't
want you to g- g- you know,

Liz Lorenzi: T- yeah

Scott: the result of this.

Just the same endpoint stroke trial,

Liz Lorenzi: It seemed like--

Scott: clinical outcome

Liz Lorenzi: so maybe an excellent
dichotomization, so the zeros and ones

might have made more sense for instant
than it did for, for distant or discount.

But discount pre-specified zero to
two and the DSMB stopped it even

though maybe zero and one looked good.

I guess I'm trying to
compare a bit between them.

Scott: Yep

Liz Lorenzi: Okay.

Scott: So, so in that, you know,
I-I-- this is how do you analyze

the endpoint and all that, but
the story is deeper than that.

And so the Discount trial, remember, was
this medium or distant vessel occlusion.

They, they analyze it, and that trial,
the distant trial, has increased deaths

and bad outcome for endovascular therapy.

It has increased zeros and ones.

It just so happened that the twos
also went in the wrong direction.

It violated the proportional
odds in that trial.

Uh, you know, a separate thing.

Now, their primary endpoint was not an
odds ratio across the scale, but the,

it, it exhibits this crossing of the
curve at some point across the scale.

So going back to more of these trials
that come out, the ESCAPE Mevo trial,

which Liz knows very well, Liz has
been analyzing that for the Step trial,

was also a trial in vessel occlusion
and looking at endovascular therapy.

And going back to that trial, which
came out in the New England Journal

of Medicine, primary analysis
was a proportional odds model.

they report that proportional
odds assumption was not valid.

It was violated, and hence they changed to
a zero one as the primary analysis because

of the violation of proportional odds.

So just medium vessel occlusion.

Again, going to this question of
medium vessel occlusion, is there

benefit of endovascular therapy?

And they saw a violation
of proportional odds.

By the way, that trial showed it
increases in zeros, increases in

deaths, increases in fives and sixes.

It was not statistically
significant on zeros and ones that.

It was actually slightly worse on zeros
and ones, but it did have more zeros

within that, and that trial showed it.

Now, I struggle with a trial where when
there's a violation of proportional

odds, we analyze it with a dichotomous,
and we'll come back to that.

But there was another trial with a similar
name to ESCAPE Mevo called ORIENTAL Mevo.

And this is also a trial that was looking
at endovascular therapy compared to

medical management in medium vessel
occlusion, the Mevo part of this.

This trial's primary endpoint was zeros
and ones in it, and they report, by

the way, that violation of the pro-
Proportional odds assumption precluded

the use of the shift, it's an odds
ratio, in the modified Rankine score.

Func- oh, so functional endpoint was used
as the primary outcome, so zeros, ones,

and twos, and they showed statistical
improvement in zeros, ones, and twos.

The violation of the proportional odds is
largely because deaths four, fives, and

sixes had no change based on treatment.

Death was actually slightly worse, they
had no change, and then you see a large

change in zeros and ones and twos.

So the proportional odds was violated

Okay

Liz Lorenzi: what was the
conclusion from OrientalMevo?

Like, what was their final readout

Scott: So in that paper, uh, in
Oriental Mevo, uh, I don't remember

where this is, JAMA or New England
Journal, but the conclusion was

thrombectomy led to a greater
likelihood of functional independence

a greater likelihood than
medical management alone

Okay.

It also says, but it also had a higher
risk of intercranial hemorrhage.

That's the bleeding, that's the
potential of, uh, downside within it.

Okay, so they refer to this
functional independence and,

and that's the conclusion of it.

And again, remember going back
to DAWN, large vessel occlusions,

DAWN was amazingly successful.

No proportional odds issues, huge
benefit of endovascular therapy

in large vessel occlusions.

And, and remember, this therapy is you go
in somebody has a blockage in a vessel,

and you go in with a device, and I think
you go in through the leg actually, and

they go all the way up into the brain,
and they mechanically remove the clot.

Liz Lorenzi: Mm-hmm.

Scott: You can imagine that this is a,
an invasive thing to do, and there are

some patients that this is probably a
bad thing to do, where you go in and

the patient, the removal of the clot,
the downstream, you know, infarct

is there and, and you restore blood
flow, which could cause bleeding.

Uh, maybe there was minor deficit and
it didn't actually improve the patient.

you caused harm by sticking this device
in in the brain and clearing the clot.

In large vessel occlusions, it largely has
this huge benefit of clearing the clot.

vessel, that's the question

Liz Lorenzi: And is there a timing
com-component to that as well?

So that time last known well
that we talked about earlier

Scott: I, I think there's a
huge question is time less?

No.

Well, you can imagine if you go in five
days after the stroke and you clear the

clot, you're gonna do nothing but harm.

And

Liz Lorenzi: Yeah, the brain…

Scott: Yeah

Liz Lorenzi: Yeah, the brain's
already dead or the tissue around it's

already dead and there's no kind of
getting blood flow back to re-heal it.

My statistician understanding.

Okay

Scott: right, A- and, you know, we, we…

You know, the, the, the people we
work with on step, all the, you know,

time is brain, and so this is a really
important part of it is, is time as well.

So there's lots of aspects o- of this.

So what, what, you know, w-
with all of that in mind, I, you

know, th- this was gonna be about
dichotomous analysis in here.

And it is a huge struggle for me that
you have this endpoint, well-regarded

endpoint of the clinical status
of a patient from zero to seven,

and you fail proportional odds

And so you analyze only the good things.

Strikes me as the exact wrong
thing to do when you have a

violation of proportional odds.

And it's a standard thing, and I,
I think it's kind of like almost

treated as a statistical thing.

Like we failed normality, so
we're doing a Wilcoxon rank test,

Liz Lorenzi: Yeah

Scott: it might be a very reasonable
thing to do, but you have this nuisance

of failing normality or you're, you
know, failing a statistical assumption.

A proportional odds is not
a statistical assumption.

I- it, it is in the model, but there's
a much deeper part to this, and I think

the worst thing to do is say, "Oh, let's
just do a dichotomous of zero to one.

Let's ignore the fact that there's
a violation of proportional

odds," which is a struggle for me

Liz Lorenzi: And when you say a
violation of proportional odds, you mean

that the odds ratio is not constant.

So we aren't seeing a shift that can be
summarized constant across the scale.

So one single number doesn't really
explain what happened in the higher

regions or in the lower regions.

Some-somewhere the, the, the result flips.

And you're saying most people see
that, and they just say, "Let's

dichotomize instead and see what happens
if we just cut the scale in half."

And even that determination
of where you cut the scale

doesn't seem to be consistent.

We've seen zero and ones,
we've seen zero to twos.

So I'm assuming this is
pre-specified in the SAP somewhere.

That's kind of how they
determine what to do.

But, um, you know, it-- Like
you were saying, this isn't some

formal statistical next step.

This is an arbitrary choice
that people have to make, and

they're putting in their SAPs.

And kind of the question is:
Is it the right thing to do?

Scott: And so you, you said something I
think that was, that was really valuable

there, and I wanna go back to this.

Uh, so, so let me, let me, uh, ask you
to do something, uh, uh, bit strange.

Uh, you know Tammy well, my wife Tammy.

listens to each of these episodes.

So explain the proportional
odds Tammy across this scale

Liz Lorenzi: Okay, so the proportional
odds ratio, we have a six…

well, seven-point scale here,
ordinal scale, and we're trying

to summarize the shifts across the
scale from one group to the next.

If you think about this in a simple
logistic regression way, you just have

a single dichotomy, and you're just
saying how many-- what's the odds of

shifting to one category or the next?

But in an ordinal outcome, we're trying
to capture how you shift across the scale.

So it's not just zero to ones,
but it's zero to one, one to

two, two to three, et cetera.

And the assumption in this model is
that we ha- we're assuming-- we're

learning a single summary so that
that summary represents all shifts,

so it's constant across the scale.

Did I do okay for Tammy?

Scott: well, we'll, we'll, we'll ask her.

Uh, yeah.

I, I, thought it was great.

So interesting in what, what the
proportional odds is doing is saying

the odds of a zero is the same as
relat- for the treatments relative

to a zero one, so across the scale.

Liz Lorenzi: Mm-hmm.

Scott: you can test whether that's
a reasonable assumption in the data.

And when that's violated, so we don't
have the same effect of treatment across

the scale, you pick a single point
and say we're gonna summarize it only

at this particular point then above.

It's, it's masking the learning
in the trial is that you have

lack of proportional odds.

Now, there are some violations that
aren't that interesting, where the

treatment is beneficial across the scale.

It's just really beneficial
and a little less beneficial.

But we're, we're addressing the, the
critical violation of proportional

odds, where actually it crosses.

So you increase certain outcomes,
but you decrease other ones, and it

could be you increase death, but you
increase perfect neurological outcome,

and that's what we're seeing in this
population across multiple trials.

So I was reading Discount, and I read
you the result of Discount, and this is

no, uh, this is no criticism of JAMA.

JAMA's that trial where it
did not improve outcomes.

Uh, within Discount and thinking about,
okay, now they're doing this dichotomous

of it, real learning here is there's
lack of proportional odds in Discount.

Within this population of medium vessel
occlusion, the lack of proportional

odds, the last thing we should do is try
to summarize medium vessel occlusion.

Say, does endovascular therapy
work in medium vessel occlusion?

The lack of proportional odds says no.

That, that's a dumb question to be even
asking at this point, because what is

the lack of proportional odds means?

It means some patients you treated,
you increased the zeros and ones.

That's good.

You increase.

You increase death, you increase
vegetative state by endovascular therapy.

To try to now say, what was the single
result of that trial, it, it's almost

like the answer is already there.

It's the lack of proportional odds.

It means we don't know the ans-
it, it, the answer's not the same

for everybody in the population

Liz Lorenzi: Mm-hmm.

And I think going back to our clinical
discussion, you can imagine if you

have a device going in to grab a clot
that, you know, there might be certain

clots that are better to retrieve using
endovascular therapy versus others.

And so, you know, it's not
that unexpected that we see

heterogeneous effects in these trials

Scott: Now an, an interesting
question would be, what you now

know, if you were in that case,
would you take endovascular therapy?

And I, I, I think you
could summarize that.

And I think, uh, you know, I would not
dichotomize it because I think things

on the bottom part of the scale are
really important, uh, within that.

And a weighted utility across the scale
is really the only way to answer it.

When you have lack of proportionality,
you have to weight how much do you

weight the good relative to the bad.

I, I think there's no way around it.

And that was gonna be part of the point
of this podcast, but that-- I don't want

that to be the point of the podcast.

We, we have other podcasts that say
the same thing, uh, within that.

The point is the lack of proportionality
is the result in these three trials,

and it's that, uh, you know, the way
we do trials now, that's the problem.

We don't need somebody else doing a medium
vessel occlusion trial where they lump

all medium vessel occlusions together and
try to create one result from that trial.

I, I, I, I think we're
doomed to get the same thing.

Now, I also don't think we need to
go in and take a very narrow, narrow,

narrow subset of maybe the biggest
medium vessel occlusions, run a big

trial and see, "Oh, that's positive.

Okay now where do we go?"

Now, now we've got, you know, 95% of
the other medium vessel occlusions.

So I think in some way it's a
failure of our trials that we run

trials where we get a success or
a failure of the primary endpoint.

I feel like that's just
completely the wrong approach here

You're, you're, sh- so,

Liz Lorenzi: Sorry, I'm shaking my head.

Scott: on audio, for those of
you only on audio, my, my host is

shaking or, uh, you know, nodding
yes, in complete agreement of that.

Okay.

So Liz, we are involved in an effort
that I think is trying to do this.

Um, and so do you wanna say a
little bit about the STEP platform?

Liz Lorenzi: Yeah, sh-- of course.

Um, so STEP platform, one of the
big efforts is to try to learn

the population of patients that
will benefit from endovascular

therapy versus medical management.

Um, and so the question isn't in a
per population, is EVT significant

versus medical management?

The question is, in a broad
population, what patients benefit?

And how we determine what patients
might be over multiple dimensions.

So it might be using time last known
well and the size of the stroke through,

you know, NIHSS, um, common stroke
scale determinant of severity of stroke.

So maybe we wanna say we wanna learn
across these dimensions what patients

benefit, which patients don't, and
can we be clearer on guidelines

of how to treat these patients.

Um, so that's, I think, the general idea.

And there's a lot of different
variations that we've looked at.

So different types of strokes
might have different dimensions

that we wanna learn over.

Um, but the general idea is really
kind of learning this enrichment

question, who benefits, who doesn't

Scott: Yeah, which, which makes it, i-if,
if we're to embrace that in this place

of heterogeneity of treatment effect,
which, which I think we've got clear

evidence from these three trials, clear
evidence, and the lack of proportional

odds on this-- the thing tells us we, we
don't know who to treat, and some benefit

and some are harmed by this treatment, is
we need different trials, and the trial

needs to come out where it draws different
conclusions potentially on patients.

And so the result of the trial might
be types A look like they benefit

and types B look like they're harmed

Liz Lorenzi: Mm-hmm.

Scott: within that.

So when you're doing operating
characteristics for this trial, and

Liz is simulating a lot of these trials
where we're trying to do it, the-- it's

not how powered are we in the trial,
because power isn't even a question.

It's what fraction of the patients
do we get right, is, is the

way to think about this trial.

So if we run a trial and we, we, we
hypothesize that a third of these patients

are harmed and a th- and two-thirds
are benefit, and we create a s- a

scenario and we simulate a trial, what
fraction do we get right in that case?

Uh, what's the probability for a
certain type that we hypothesize

benefit that we say they benefit?

So there are aspects of power, but
the trial's very, very different,

and as we learn the result in one
subset, we stop enrolling that, and we

might continue investigating others.

The trial has to be so different than
these one-size-fits-all when we know

the answer is the question is wrong

Liz Lorenzi: Yeah.

And the other important part
to that is we're not just

cherry-picking across a space.

We're trying to use
information we already know.

So there has been tons of these
stroke trials where we know

where there's benefit, where
we know there's not benefit.

So we can incorporate that into the
framework so that there are parts where

we, we might have a higher chance of EBT
bene- being beneficial a priority prior

to any data being collected, just because
we know patients that were seen a few

hours earlier were also, um, saw benefit.

So there, there's a cool Bayesian
prior aspect to this as well, as

well as trying to make decisions
that are somewhat logical.

So, you know, you don't want a
decision in the middle of that grid.

You want the decisions to move maybe
in a, a direction that's first…

We like to call it monotonic.

So they're moving in a direction
that are, is consistent.

So, you know, strokes that are
within this size core volume should

be treated, um, and not just treat,
you know, one fifty core volume.

Um, so that's another aspect
of it, is trying to keep the

ordering of the covariates that
we're interested in as part of it

Scott: Yeah.

Uh, it's, and it's a very
cool trial and it's nice.

It's wonderful, by the way, this
is NIH-funded, uh, trial, which,

which makes a ton of sense to
ask this question, who should and

shouldn't get endovascular therapy?

But the trial design parts
are, are, are very cool.

Uh, the Bayesian part you
described to that, very, very cool.

I also think this issue is going to
show up in many, many diseases, whether

this is heart failure, whether this is
Alzheimer's, uh, where when we learn more

and more about the disease, rather than
these large trials that get one answer

for the population, we're gonna find more
and more benefit some and not others,

we're gonna have to figure out how to
figure out who they are these trials

Liz Lorenzi: Yeah, I was thinking
back to the oriental NIVA one.

You know, that-- I think that trial
seemed to get it a little bit more

correct in that they did dichotomize.

They did take, give a conclusion
of, you know, we saw more functional

independence in EVT, but they also
said that they saw more bleeding.

So I'm curious from a clinical
perspective, do you know who

to treat from that conclusion?

Or…

'cause, you know, on one end
you had more bleeding, on the

other is you improved outcomes.

So it's back to that kind of
functional issue and I don't know.

I thought that-- I, I, I just think
the general i-- dichotomization, the

issue, you're just masking what might
have happened later in the scale or

in, to different parts of the scale.

Scott: the risk scale.

Liz Lorenzi: And I guess in this
one they at least tried to capture

that by giving a two-part answer.

But I don't think it's a very
satisfying answer of I, I don't

know how to action that result.

You know what I mean?

Scott: Yeah, and, and, and so some of
the-- A- and it's interesting because I

almost read it as we saw clinical benefit,
but we did see some safety issues.

Liz Lorenzi: Mm-hmm.

Scott: Where some of those, uh, bleeding
things, the severe ones lead to poor

neurological status, maybe even death,
which is the worst neurological status.

But so across the scale, when you're
seeing this increased deaths, you're

seeing increased bad outcomes.

Within that, I feel like this
has to be reported and I struggle

with see better clinical outcomes.

Y- y- you know, w- within it being
the outcome in a dichotomous, when

it's only dichotomous in that.

But you're right, it did at
least add that part to the story

Liz Lorenzi: Yeah

Scott: Okay.

So, um, ha- have I, have
I gotten this straight?

Liz Lorenzi: I think so.

I mean, I-- and I think maybe something
to come back to is, like, we don't…

And, y-you know, it's not an
easy problem as a statistician to

pre-specify an analysis plan when
you don't know what will happen.

Um, you know, I think this
is an ordinal outcome.

It's, it's reasonable to
suggest a proportional odds

model as your first analysis.

Um, I guess one thing to think about is,
you know, if you get the data and you

don't see it proportional, if we don't
think dichotomizing is the right idea,

like, what is the right thing to do there?

Scott: I, I hope that was partly
rhetorical because I, I, I am

going to agree with you that
this is a very hard problem.

when you get the lack of proportional
odds, you have to address that question.

You have to address, is the benefit
on the part-- the left, the, the good

outcomes outweigh the harm on the right?

I mean, you, you can't sweep
it under the rug by only saying

increase excellent outcomes.

Uh, and I think that's the wrong answer.

Liz Lorenzi: Mm-hmm.

Scott: Any single dichotomous analysis
when you have lack of proportional odds,

I think has to be the wrong answer.

When you described this to Tammy,
you said that proportional odds

means we largely have similar
effect across the whole scale.

And you say, "We don't have that,
so I'm gonna summarize it a single

ti-- at a single point across the
scale if there's benefit or not."

Liz Lorenzi: Yeah

Scott: think there has to be a
clinical weighting of those outcomes.

it's a problem we're stuck with.

We have to do that.

And I think doing that ahead of time
and, and the, the, um, the weighted

modified Rankin score, the w-- uh,
weighted WMRS, is a way that does that,

and it looks at both economic impact
and patient preference impact, was a

way to prospectively do that in a field
where we expect proportional issues.

You have to do it.

You cannot escape that
question of, uh, of it.

And I, I, I, I think our current, our
current structure of clinical trials

where, uh, it's overly do we get a
result that's a positive result of

the trial leads us to do one thing,
where I don't think it's tied to

necessarily the right clinical thing
or the right thing to a clinician out

there that has to make a decision, or
a patient that has to make a decision.

And I worry trials are failing that

Liz Lorenzi: I agree

Scott: And I'll, I'll lump us in
there, uh, uh, uh, in there, uh,

within it, uh, and, and recognize
it's a, it's a hard problem, but I

think we, we're, we're working on that

Liz Lorenzi: We are, yes

Scott: We, we are in the interim of
this science thing and especially in

acute stroke, uh, and we're, we're
getting better and we're working on it.

thank you, Liz, uh, for joining
me and keeping me straight and,

uh, hosting, uh, this episode

Liz Lorenzi: Of course.

Thanks for having me.

It was a fun conversation

Scott: And appreciate everybody out
there for, for listening, for joining,

uh, discussing the proportional odds
assumption and what it means and doesn't

mean, uh, within clinical trials.

Until the next time, will
be here in the interim