Statistical Methodology Groups in the Pharmaceutical Industry

In this episode of "In the Interim…", Scott Berry is joined by Jenny Devenport and Kaspar Rufibach of Merck Group, co-authors of the EFSPI Statistical Methodology Leaders paper "Statistical Methodology Groups in the Pharmaceutical Industry," and May Mo of Amgen, a co-author of the commentary on the paper from the Statistical Innovation Community, an ASA Biopharmaceutical working group. The guests describe methodology groups within a pharmaceutical company as a strategic investment, and innovation as the product of invention and commercialization, where publication, regulatory dialogue, and validated software ("statistical engineering") turn new methods into common practice. They discuss why new mechanisms of action and therapies such as CAR-T need more than a t-test or a log-rank test, and how estimands, causal inference, adaptive designs, and external data borrowing are reaching clinical trial statistics. The conversation addresses the concern that a dedicated group divides methodologists from project statisticians, how these groups demonstrate their value, why quantitative decision-making is a core competence of a pharmaceutical company, and why curiosity and collaboration matter as much as technical skill.

Key Highlights

• Innovation as the product of invention and commercialization, and the role of methodology groups in scaling new methods.
• New mechanisms of action, from delayed immune activation to CAR-T, that a t-test or log-rank test cannot handle.
• Estimands, causal inference, adaptive designs, and external data borrowing entering clinical trial statistics.
• Avoiding a divide between methodologists and project statisticians, and why there is "no unicorn statistician."
• Showing value to leadership, from study involvement and due diligences to gains in probability of success.
• Quantitative decision-making: presenting design options with their risks, costs, and trade-offs.
• Curiosity, collaboration, and communication as the traits of an influential methodologist.

For more, visit us at https://www.berryconsultants.com/

Creators and Guests

Host

Scott Berry

President and a Senior Statistical Scientist at Berry Consultants, LLC

What is In the Interim...?

A podcast on statistical science and clinical trials.

Explore the intricacies of Bayesian statistics and adaptive clinical trials. Uncover methods that push beyond conventional paradigms, ushering in data-driven insights that enhance trial outcomes while ensuring safety and efficacy. Join us as we dive into complex medical challenges and regulatory landscapes, offering innovative solutions tailored for pharma pioneers. Featuring expertise from industry leaders, each episode is crafted to provide clarity, foster debate, and challenge mainstream perspectives, ensuring you remain at the forefront of clinical trial excellence.

Judith: Welcome to Berry's In the
Interim podcast, where we explore the

cutting edge of innovative clinical
trial design for the pharmaceutical and

medical industries, and so much more.

Let's dive in.

Scott: All right.

Welcome everybody back to In the Interim.

I'm your host, Scott Berry, and
I have a, a, a, a good topic

today, a topic of innovation.

This podcast focuses on clinical trial
innovation, statistical innovation, and

today we are gonna talk about innovation.

We're gonna talk about innovation at
pharmaceutical companies and the role

of methodology or innovation groups.

We, we have a-- We're gonna talk
about a particular paper that was

published with commentary, and
we have authors from, from this.

So the paper is "Statistical Methodology
Groups in the Pharmaceutical Industry."

And we have two authors from that,
and then we had commentary on that,

and we have two authors from that.

And so this is going to be
our topic for the, for today.

So my first guest is Dr.

Jenny Davenport.

She is a senior director, patient-focused
real-world evidence at Merck Healthcare.

She is in Switzerland, and you can go
back to episode forty-two of In the

Interim and learn much more about her
journey from Austin, Texas, where, where

I'm recording from now, to Basel, uh,
where she is now and recording today.

And also joined by another
co-author of this paper, Dr.

Casper Ruffois, and he's a senior director
and head of Advanced Biostatistical

Sciences, also from the Merck group, and
also a part of the FS- FSPI Statistical

Methodology Leaders, which generated
the, the, this paper we're talking about.

He's also a member of the ex-
expert writing group for ICH E20,

which is a topic for another day.

Uh, and we also got to visit with Casper
on episode thirty-nine, so you can go back

and learn a little bit more about Casper.

A commentary on this paper
was written by a US group, the

Statistical Innovation Community,
and it's an ASA Biopharmaceutical

Working Group, uh, US-based.

The, the original paper was
F-SPY, a group in the, in Europe.

And this one, We're joined by Mei Mo,
who is, uh, associate vice president

of biostatistics and head of design
and innovation, and we'll come back

to that, uh, as the topic of this.

And she supports a wide range of
therapeutic areas and products there.

She's been at Amgen for 20 years,
uh, and 12 years working on products.

And now, uh, since 2018, she's
been a founding member of the

Design and Innovation Group and,
and she now leads that group.

She was at Abbott Diagnostic for,
for Abbott Diagnostics for nine

years before that, So welcome
everybody, to In The Interim

Kaspar: Hej

Jenny Devenport: Hi, Scott.

It's good to be here

Scott: Okay, so let's talk
about the original paper.

And my first question, I guess,
Jennie, to you is, what is a

statistical methodology group?

Jenny Devenport: That's a great question.

I love that question, obviously.

Um

In the paper, we tried to tackle this,
um, very broadly, but I'll try to give a

concise definition here that methodology
groups are a strategic investment,

and what they do in the pharmaceutical
industry, uh, when they exist, is

they lead the conception, evaluation,
implementation, and application of

appropriate statistical methods.

Um, a, that's a big name for a concept
that I think, uh, Casper might want

to comment on, which is how do you
innovate and produce, know, produce

results that not only change your
company's projects but the field?

Kaspar: No, thank you, Jenny, and, more
than happy to, to add an equation to this.

So maybe, Scott, that's not something
you typically talk about in the podcast.

But, uh, this is something, uh, we
came, uh, or, or we learned about from

colleagues in, in other companies.

and I think it goes back to a,
a Harvard innovation professor,

equation Innovation is the product
of invention and commercialization.

invention is what I think a lot of
statisticians do in companies, whether

they are in such a methods group or
they are leading molecule projects

or they are trial statisticians.

but if these inventions are not kind
of, um, how should, how should I

say, canalized or kind of put to a
clear purpose, there is a risk the

commercialization piece suffers.

And if you think about statistical
inventions in pharmaceutical industry,

maybe in the last two decades, some of
them really did this commercialization

piece well because the…

there was an acknowledgment
that this needs to be done.

It's not enough to be right.

It's not enough to invent something.

You also have to scale it up.

And these groups can be very helpful
in that, in kind of bundling, I think

that was the word I was looking for.

Bundling these efforts within the
company and then at some point

also across companies to influence
entire industry and also offer a

forum for dialogue with regulators.

And for that, these groups have been--
have proven to be very, very useful.

Scott: Okay, so this is, this is a
separate group within a pharmaceutical

company whose remit is, as you described,
uh, to, to do this bundling, the

conception, evaluation, and implementation
and application of appropriate

statistical methods, which is what I
think, uh, g- comes from your paper.

Which means that separate time is
dedicated to this activity as opposed

to maybe working on a product trial.

Uh, a- and that's, that's I guess what
you're, you're proposing or discussing

is that separation a little bit

Jenny Devenport: Yes.

I mean, it is absolutely possible
that a project statistician can

invent something to solve a problem
that they have on a project.

But as we all know, there is tremendous
pressure to get things done in our

industry, and whether or not they have
the bandwidth to continue to test the

bounds of their solution beyond the
immediate problem at hand, to socialize

it with peers in other companies in the
pharmaceutical industry, to publish it,

to socialize it at conferences, to go
through the peer review process, and to

get feedback from regulators outside of
the original context, that's typically

a time requirement that the average
project statistician does not have.

And so it's not that they cannot do
this activity as a function of capacity.

It is a, it is a question of how
much time they have to be able

to dedicate to that activity

Kaspar: And maybe add to that, in the
rec- in recent years, many of the methods

that have been developed in pharma, they
will not make it into commercial software

packages immediately, not even midterm.

So new methods have been developed
for which also high-quality software

that can be used by other companies
that has some degree of validation,

where there is clarity that this can be
used also by companies, by academics.

That need has grown as well, and
that is often also within the remit

of a statistical methodology group.

So kind of this broad tool development.

have even invented the term
statistical engineering for that piece.

And I think this is also, as Jenny alluded
to, imagine you're a project statistician.

You have a team running
five, six phase three trials.

have this idea, and then having
support in pushing the idea, publish

it, develop software that others
can use, I think increases the odds

that an invention is actually scaled
up and commercialized, and then the

product of the two is positive not zero
because the commercialization piece

is zero, that these odds, uh, increase

Scott: Hey, May, y-y a commentary
from your group in the United

States, uh, across the pond.

Uh, one of the interesting things
I was struck by initially, you

referred to moving away from the
term methodology, uh, to innovation.

Interested in, in why you
think that's important?

May Mo: I have to give credit to,
to, uh, my, uh, the, the head of,

uh, Amgen's Biostat organization we
call Center for Design and Analysis.

she, uh, was also the first generation
of a design innovation head, uh,

whi- like, which is my current role.

it was named by her, and the reason,
of course, I, I contribute i- in the

discussion, we do not want to focus on
methodology and the methodology only.

And team was built out of a
urgency, as Casper just mentioned.

the new way of doing trial design
and analysis need to be, um, not

pocketed with individual statistician.

And a lot of, uh, fascinating new
methodology, and, uh, it bring

efficiency to statistics and also to
trial design, is not widely adopted.

at that time, it was a top, top-down,
um, from our company, um, top leadership,

to build this group so that we can
promote, um, advocate innovations

throughout biostat department.

And then we were pulled out
to forming, uh, this team.

At that time, the name was deliberately
get away from methodology is to

push for, um, innovation and, uh,
pushing for innovative design.

So hopefully that give you some history.

And that was, uh, 2000,
uh, as, uh, you heard.

eight…

I think end of 2017, beginning of
2018, so it's nine years of journey.

Scott: Very nice.

So I-- it would be great, a-and
interestingly, the names of the groups,

uh, across here, Advanced Statistical
Sciences, Center for Design and Analysis,

Statistical Research and Innovation, all
leading, you know, very similar things.

I-- it would be great to hear a little bit
more maybe from each of you what, what,

what the makeup of the group is, general
size, and, you know, how much time is

spent perhaps on innovation methodology
as opposed to maybe product work.

And, and, and I'll let you
all sort of describe that.

And let, let's, let's, uh, uh…

So, May, why don't you describe your group
a little bit, and what does your group do?

May Mo: Okay.

uh, my group, um, keep on expanding,
um, and I can explain why.

At first, our focus is
very much on trial design.

that was our first, um, like, companywise
ask us to give laser-sharp focus on.

So we push for, uh, Bayesian, um,
design, adaptive design, and, uh,

we're looking for opportunities to
leveraging external, uh, available

data, be it it's a historical control
or real-world data, we're pushing

for, um, potential extrapolation.

Um, so all those things, um, was the
focus in the first couple years, but

at the same time, we start building
other specialized, um, capabilities.

So, uh, to name it, centralized
statistical monitoring, um, safety signal

detection, um, precision medicine-related
exploratory biostats, um, because

emergent HTA, we start focusing on,
um, indirect comparisons, network

meta-analysis, MAIC, um, to name a few.

And I can see these, uh,
will continue to build.

So we more and more become a specialized
team focusing on those area uh, the

capability may be not concentrated equally
into each of the product or TA team.

So we become a true partner to, um,
statisticians and working on many

things, and they need our help.

So it's really, um, together.

So as of now, the team is about,
um, like between 40 and 50.

I no longer really exactly know the count.

and supporting, um…

We become a large platform, uh, function,
sits bes- uh, behind biostats, supporting

every team, every product, every study.

Jenny Devenport: So I want to react
to that a little bit, if that's okay.

There's a couple of really
interesting things that you said.

Um, uh, one of them was about this
topic of innovation and, and something

that we tried to address in the
paper is why we need to innovate?

You know, why wouldn't you just run the
same kind of trial every time if that can

be operationalized faster and lead to a…

You know, assuming the drug works,
then, you know, all is well.

And I think there were a
couple of things that happened.

Um, one is that we started, as an
industry going into therapeutic areas

and into mechanisms of action that
required fundamentally different designs.

You know, um, when you talk about
delayed onset of activating the immune

system, you start to have, you know,
very, very strange effects going on in

your data, you cannot just do a T-test.

Uh, you cannot just do the same thing that
you've always done in terms of design.

And so this notion that, um,
it's not just that we're adapting

statistics for its own sake, it's
to meet the demands of the business.

And when I hear you talk about the
specializations that you have at Amgen,

it is because there is a real business
need driving that specialization.

And I've heard other colleagues from
other companies comment that it's

very difficult for one person to keep
on top of every single innovation

within our field, and so you may
need to create some redundant systems

Scott: So, so this, th-

Kaspar: I, I

Scott: this seems to be a huge premise
of all of this, that we can't keep

doing what we were doing 10 years ago.

Uh, if we could, there'd be no
reason to innovate methodology.

So you're at, you're at companies where
the drugs being produced, the biologics,

the treatments are incredibly innovative.

They're, they're, you know, so much
different than they were 10 years.

Are we innovating the clinical trial?

Are we innovating statistics
to keep up with all of this?

So if you reject that premise that we
can just keep doing what we were doing

10 years ago, we don't need methodology.

So I assume you have
to accept that, Casper

Kaspar: Yeah, of course.

And, uh, you can add to what Jenny was
mentioning, for example, um, CAR T.

Again, requ-- I mean, there are
so many interesting aspects, not

to say challenges, when you want
to design a trial like that.

And, um, there have been big
methodological developments that

are now reaching clinical trial
statistics because I think…

I mean, I, I started in this industry
about fourteen years ago, and, uh,

you could maybe get quite far if
you knew what a log-rank test is.

th-these, these times, I think
have dramatically changed.

So think about it came through this ICH
E9 R1 addendum on estimands, but now

I think they're-- the clinical trial
statistics world now just becomes aware

what causal inference has to offer, uh,
to answer questions in a more s- kind

of rigorous way than maybe in the past.

That's one thing.

Uh, Scott, you mentioned, uh,
adaptive designs, trying to

make designs more efficient.

another piece, of course, is borrowing,
whether you do that in a b…

Some are very fond of doing this in a
Bayesian way, but you can potentially

do that also in a frequentist way.

I think the p-- the, the key thing
is how can we synthesize from

different sources to make, decisions?

this requires statistical knowledge,
and this even when you are in

such a methods group, even…

And, and maybe are a bit less involved
in the daily business, in molecule

projects, then, it is very difficult
to keep abreast and, and stay on top

of these things as an individual.

But of course, as a group,
that is then the remit and

the aspiration what, uh, what

Scott: Mm-hmm.

Kaspar: have.

Scott: So, so interestingly, it could
be a discussion of what is a better

approach, Bayesian approach to doing
external data, frequentist approach?

You brought that up.

We could have a long discussion about
that, but I think that's almost your

point, that comparing these, the pluses
and minuses, all of that is part of the

methodology group's role in this, is when
should I do what, and then sharing that

throughout your companies and externally.

Is that a fair summary?

Kaspar: This is very important, and
now I mention that even if you are

a statistician in a methods group,
it's not easy to make sense of

all these things that fly around.

Now, imagine you are a colleague in a
regulatory department in a company or a

clinician, and you get all these terms,
these methods, you see it at conferences.

I think that that's also where these
methods group and have to play a big

role as a catalyst within the company.

So for example, here at Merck, we
are doing what we call stats boosts.

are ninety-minute sessions.

They're open to all of R&D.

We typically, on average, we have
about a hundred people attending, and

we are now at session twenty-seven.

So Scott, we will catch up within the

Scott: Yeah.

Yes, yes.

Kaspar: at some point.

Um, I, I also liked a lot what, what
May said, kind of growing the group,

and this is also the ambition we have.

Maybe let me call it to spread the
quantitative gospel across the company

because many functions in these companies,
not just in R&D where clinical trials

are run, quantitative questions and
are more and more trying to embrace

modern quantitative methods, and that's
also where these groups can play a

Scott: Okay, so let's, let's…

Jenny Devenport: maybe to-

Scott: No, go ahead, Jenny

Jenny Devenport: I was
just gonna tag onto that.

I know some of our other commentators
who've been in the industry a long time

that they hoped that methodology groups
go beyond clinical trials, uh, would

look beyond the opportunities of just
clinical trial design to, uh, impact drug

development in a more fundamental way.

And I think that's where you get into
fields like quantitative decision-making

and, um, you know, some of the, some of
the monitoring that happens and some of

the longer term, you know, end-to-end
kind of, not just decision-making,

but looking at the entire development
cycle and how can we shorten it.

How, you know, what, what
opportunities do we have bring, uh,

new medicines to patients faster?

And not all of them are within
the realm of trial design.

Some of them are in program design,
some of them are in operations,

some of them are in manufacturing.

there's just a lot there.

And so, again, it's not that
methodologists need to take over the

world, but there's some fundamental
places where we feel that we can add

value, and we should be open to that

Scott: Okay, so let's, let's-- I, I
guess the paper is largely laying out,

um, what a methodology can-- methodology
group does, its role, its impact.

A big question might be,
should a pharmaceutical company

have a methodology group?

Steve Snappen did a commentary on it,
and he says that within his company,

he doesn't have a methodology group.

And he points out a couple
potential negatives.

So maybe, uh, I, I'll play the role
of Steve Snappen here, and you,

uh, positive or negative reaction.

One is that he thinks that it may create
a bit of a division between a product

statistician and the methodology group.

Also, that to some extent, some of--
he thinks some of the fun of being a

statistician is methodology development,
and it's a remit of everybody in

the company to do methodology.

And if the group is doing it, am
I not supposed to do it, or am

I, you know, not part of that?

So are there negatives of
having a methodology group at a

pharmaceutical company, in your views?

Jenny Devenport: I can
talk about this all day.

I'll let

Scott: Yep.

Jenny Devenport: else start.

Scott: No, you see, you start Jenny

Jenny Devenport: Okay.

Well, I mean, I see his point,
and that's certainly, I think, a

view that an executive might take.

But what I have seen in
practice at several companies

now is very, very different,

the good methodologist relies on the
project statistician to understand the

business context in which a trial or
program is being conducted, and interact

with them the whole way through to
assess whether, you know, what is the,

the problem that we need to solve?

What are some of the potential
solutions we could offer?

And so it's the methodologist
and the statistician working

hand-in-hand all the way through the
commercialization period at times.

It's just that that project
statistician doesn't have to

do the heavy lifting alone.

neither then does the methodologist
in terms of having to have the full

context all in their head on their own.

And I think this, this collaboration
piece is something that we really tried

to emphasize as a way to ensure that
you don't have this negative perception.

And again, at the companies that I
have worked for, where I have been

involved, this collaboration is very
good and, and leads to a very, um,

a very positive relationship between
project statisticians and methodologists

Scott: Jasper?

Kaspar: Uh, Scott, you mentioned
one point, of this division.

this is, um, a mentality piece,
and is part of the job profile

of a statistician in that methods
group, this genuine interest in the

collaboration with the business.

This is-- If, if you are not
embracing that to a hundred percent,

maybe you're better off at the
university because th- it, it's not

an ivory tower within the company.

Absolutely not.

All the problems that you work
on should come from the business.

Sometimes there may be you investigate
something like what you said, you

compare some methods, but still
that is motivated by a true business

problem or an industry problem that
comes out of an industry group.

And then the other thing that,
uh, that you mentioned is kind of

the methods are the fun, and it
should be the remit of everybody.

Yes, yes, yes, absolutely.

It should be the remit of everybody.

And th-- But then maybe in order to
scale things up, to learn more what

other companies do, to understand what
does the literature say, maybe you have

somebody who is, um, in the methods
group, who is maybe a specialist on the

topic, and then you can, uh, join forces.

Um, and the…

my last point to what Jenny
said, the heavy lifting, I guess.

So I was at Roche, uh, in my previous
company before there was a methods

group, and then there would have been
a lot of heavy lifting that should

have been done, but it didn't happen
because there was not such a group.

So there is-- this heavy lifting has
to be done, and for that, I think a

Scott: Hmm.

Kaspar: helps a lot

Scott: May, how about at Amgen
or, or in your experience?

May Mo: I, I definitely,
um, see both side.

So where Steve mentioned, um, can
be some statistician's concern.

So we have to recognize that,
like, from, from the innovation

methodology team from here.

So there, there are several things.

Um, people wanna make sure not only the
methodology group can do methodology,

not the only innovation group can do
innovation, Um, statisticians actually

inside the door and outside the door
of, uh, this, uh, innovation, group

separation, like, uh, uh, or division,
are trained the same originally, right?

they have different interest, and some
people want to get into the direct project

work seeing broad, um, of problems.

some statistician like the methodology and
wanna drill deeper and become specialist.

Um, I want to say, and, um,
I just wanna say two point.

I could talk about this all day, too.

Um, one is I deliberately set our team
with not a, like, stonewall So you can

talk about many of my team members moving
back and forth between the, the so-called

innovation group and the product team.

Myself was a trialist, right?

And many of our, um, design
innovation, leaders were trialist.

And I, I say this thing to my team all
the time, "You don't be a car designer

until you can actually drive in cars."

So, um, so we know the purpose,
we know the real problems that

help us to talk to the teams.

And when we have the new generation
of innovation statisticians coming

from universities, I keep them
30% on trials, and they have

even better exposure to trials.

They, they work on phase
one, phase two, phase three.

They work on oncology, non-oncology,
cardiova- I mean, cardiovas-

uh, and, uh, inflammation.

You name it.

They try to do the exposure as
much as possible, so it's a mix.

First point, there is no solid wall.

It's, it's a group actually, um,
really mixed with each other,

just with different focus.

is, um, how, we avoid, um,
the vision of, uh, um…

to, to me is, uh, um, engaging,
um, statisticians to work together.

I use analogy, know if I can mention here.

I use it a lot recently.

Um, thinking about the f- the
profession that we work very closely

with, the doctors, I get that.

I real- I use that because last year
I had really, um, experienced that

a lot because when my, my father was
really, ill at hospital, his primary

doctor is a nephrologist, but daily
I have to see cardiologist for his

heart problem, and he delev- developed
cancer, so I have to see oncologist.

I have to see many doctors, um,
endocrinologist, all those people

together to solve very complex
health, health problem for him.

And I saw those doctors working very
closely together, and all of a sudden

I realize our product statistician's
almost like my, my dad's primary

responsible physician, and he
reach out to all those specialists.

There's The reason the doctors can
work so closely together is, one,

patient's the most important thing.

Two, they realize their profession
is already advanced so much,

there's no one can be a,

Good at all, let's just say that
that would be a unicorn, right?

There's no unicorn statistician.

So if our professional can be enough to
know we don't know any, like everything

anymore, we really want to work together
to, um, advance the product we love,

just like our patient we care the most.

So with that, I use that explanation
to the team, and people see that

and hopefully that can change a
little bit how people work together.

Sorry, take long.

Scott: No, no, that's great.

And, and so a huge part of what
everybody said is that you're

solving the, the puzzles, the
problems that your companies have.

I don't come in with my
research agenda and I…

It's publish or perish and, you know,
it's, it's not about my research agenda.

It's I'm solving problems.

So at the, at the other end, when
you have a methodology group and a

pharmaceutical company is investing in
this methodology group, can-- do you need

to, do, are you asked to justify that
you've provided value to the company?

This might be a hard thing to see
that, okay, we've spent a certain

amount of hours, a certain amount of
resources have gone into methodology.

Can you help identify
that that has helped us?

We've run better trials.

We've run the right trials.

We have, we haven't run bad trials.

You know, I…

Is, is that a hard thing to do?

Do you need to do that?

Are you asked to do that?

Uh, uh, Jenny

Jenny Devenport: I mean, absolutely.

I think that's, that's a fundamental
question, especially when you're,

um, you know, a dedicated group.

Uh, is always asked in any function,
you know, what value have you added?

And of course, um, in the realm of trials,
drug is either gonna work or it's not

going to work, and the next blockbuster
is probably not, uh, decided on the basis

of, of minor deviations in trial design.

We can debate this, but, but if it's
really a blockbuster, the fact that you

have 60 patients versus 70 patients may
not be the difference in what you see.

Um, I've certainly worked on, on, um,
programs where, you know, the, the

difference, uh, in terms of efficacy was
extraordinary and, you know, one could

argue that one have done the program
with a much, much smaller sample size.

We didn't know.

But I guess what I'm saying is that
you must always be able to articulate

what you do, and you have to do
this in an environment where what

you're testing may not be successful.

So how do you quantify or, or characterize
the value that you have added?

And some of it comes down to
decision-making, to pointing out the

areas of uncertainty, to enabling
people to make the right decisions about

progressing programs forward or not.

What can you do to reduce risk?

Kaspar: To build on that, uh, paper,
we have this statement that the core

competence pharmaceutical company
is quantitative decision-making.

And as soon as you can support that
activity and take the HiPPO out of

the room, the HiPPO is the, highest

Jenny Devenport: I stayed because,

Kaspar: pay-- hi-highest paid person's
opinion, uh, as opposed to relying a

decision on data, then that is one metric.

But talking about metrics, I
think in this context, metrics

are It's difficult to quantify.

Uh, I think it's-- it needs some, uh,
support from biostatistical, biometrical,

or even higher above some appreciation
that such specialists can bring something.

And these specialists need to make
sure they earn that appreciation,

through the methods piece, but
also through the influencing

piece, through communication piece.

Very often, I mean, we all have been--
we, we started as drug developers

in, in clinical trials, and very
often even there, the quantitatively

trained people are also often those
people who are analytically strong.

They can tear apart the big
problem, all the small problems,

and put it back together.

That's what you learn when you are
a mathematician, and these are all

aspects of that role that hopefully,

are clear to decision-makers in companies.

But you have to count certain things.

How many due diligences you contributed
to, how many trial designs you influenced,

and then there is this whole outreach
piece, which is again, soft to capture.

but that's part of what you have
to do as a group, this, this, this

lobbying and making sure that people
understand the value that you create.

And here it comes again, this
collaboration with the business.

Because if you would work in ignorance
of the business, just pursue your

own projects and nobody cared on both
sides, then of course, that would

start to be difficult at some point.

But you support the project statisticians,
the trial statisticians, if you

support the managers in the statistics
function in solving their problems, in

addressing their problems, in showing
flexibility, jump to rescue if you get

an FDA question and three days later you
have to send, send in a response, and

that's where you can really contribute.

Scott: May

May Mo: Yeah.

I, I can say that's the first thing,
uh, when we build a team, uh, we

were already asked to do, um, is to,
um, have a matrix to show impact.

Um, at first the, the matrix
focusing on, um, like how many

team worked with us, right?

Like of our products and studies.

And of course, there's a longer
term, is whether the final design,

by design feature show, um, impact.

Let's say shorten the, the trial
decision saving number of, uh,

patients or, um, be able to claim
a broader uh, uh, population.

So all those link to future potential,
um, financial return as well.

So instead of MPV or ROI, um, it
become a probability-based adjusted.

So statistician can provide all that.

Um, I have to say, um, team actually,
because the setup, we were quick

to a, a huge success, probably
due to top-down support as well.

So the, um, the product team involved
us, number is in, in the 90%, the studies

that we involve in, in phase two and
phase three above, and we actually get

into some phase one, um, as well because,
uh, for oncology, some phase one trial,

it can quickly, get into like a late
development So, so that's one thing.

And the other thing we did is, 'cause
as- after, like, just two years, we

already plateaued, and I, I don't think
we need to really pushing for 100%.

It's a fee for purpose.

So 70 to 90% is good enough,
um, involvement in studies.

nowadays, I, I don't see,
'cause I do sit in, um, global

development, uh, governance body.

So for all the trials, um, come in program
level or study level comes through, I,

uh, rarely see any, program level like,
um, so, uh, sequence of studies, right?

Or, or study design without touch
of, uh, the design innovation team.

Um, the other way to, to broad,
uh, awareness, um, Amy, um, the

head of Biostats, uh, work with
development co-host internal advanced

innovative, uh, design series.

So this series, uh, pretty much happen
every month or every other depend on,

frequency, involve people from outside,
and recently a lot of, uh, highlights of,

uh, uh, innovative approaches from inside.

It could be methodology, it could
be design, it could be execution,

could be many innovation enabled
by the quantitative science.

um, that generate broad awareness.

we always, uh, using those and keep
portfolio of our impact to show

these are the things, uh, we worked
with team together and achieved.

and we also just like,
uh, I, heard, that the, um

Casper, you mentioned, uh,
is that the morning boost or,

uh, stat boost, um, program?

We also have something, uh, more than,
20 years I'm here, always have it.

It's Biostat Lunch and
Learn, and give lessons.

And we recently refreshed it, uh,
with a statistician from my team.

So we have, uh, monthly lunch and
learns giving to the entire company

at regulatory, um, development
and, you know, operation.

um, um, people understand why
the stats principle are needed.

right now we refresh the co-- uh,
curriculum, which was static for a long

time, we added those, the innovation part.

So it become a fifty-fifty percent mix.

Hopefully, that can also
generate education throughout.

So I just wanna share,
uh, the impact side.

Um, and lastly, I think, uh,
I did make my team to continue

do time coding into project.

People wondering whether a methodology
team need to do that, and I did,

'cause the specialist team are
not just a behind-the-scene,

um, sp- uh, methodologist.

They sometimes are practitioners,
like the, the specialist, uh,

oncologist or cardiologist.

so like HTA, safety
signal detection, right?

And our design work.

We are actually-- They are actually
working on real study, real product.

So, with that, team actually can
show, um, we are greater than,

like, eighty or ninety percent
billable hours, let's just say.

Th-those are on products and on studies.

they are either working on the
current problem working on slightly

future problem of the products.

So, um, that also help
to highlight our impact.

So I just wanna share this much

Kaspar: Thank you, May.

And, and maybe Scott, if I may,
for the listener to kind of two

thoughts that I also often use
to make this concept prominent.

you have a portfolio of drugs, and you
increase the probability of success of

every project by one percentage point.

That's dramatic, the money
that's at stake in our industry.

And the other way of looking at it,
you have a methods group they change

one decision about you running a phase
three yes or no to the better, then you

have funded the methods group for the

Scott: Ja.

Kaspar: century.

um, are also kind of…

But of course, that impact, that
credibility, that needs to be there and

that needs to be established, built, and,
and maintained, uh, within the company

Scott: Yeah, so I, I think that's,
that's a huge part that this, this

group is gonna change much more than
trial seven's design, that by-- You

might even be able to do quantitative
decision-making, that you never run a

trial on something that's unlikely to be
successful, meaning you take a shot on

goal on something else that is successful.

And in the counterfactual of not
having a methodology group, you don't

even run the trial, you don't even
investigate the drug, or you, you, you

spent too much on other trials that
you couldn't take that shot on goal.

So thinking about this, if I were trying
to decide is a methodology group a good

thing to do, I'd have to do a cluster
trial where we randomize companies to

have a methodology group or not, and
you'd have to look at the impact 10 years

down the road at the, the company level.

You couldn't look at a single trial
level and all of that, and I think

that's all of your points, that you're,
you're gonna raise everything up by the

quantitative decision-making, by, by this
i-innovation and methodology being done

Jenny Devenport: Absolutely.

I think, uh, you know, we tried to address
a little bit about the fact that different

organizations implement this and I would
encourage listeners to go and to read the

paper and, you know, engage in the debate.

And so maybe to zoom us back out a
little bit, you know, one of the things

that was exciting to me about this
becoming a discussion paper is that it

broadened the discussion, um, much more.

Um, that…

A- and one of the points that's
important for me is encouraging

debate in global efforts.

what you see here is FSBI and ASA
are engaging with their counterparts

in other companies to strategically
enable the advancement of the field.

It's not just a professional society
that gets together and talks.

It is a group that is, that…

are groups that are aimed around
common goals in science to move the

field forward, and I think that's a
really critical piece that is enabled

by papers like this, by groups like
these, that are willing to discuss

and share what their priorities
are and what they're working on.

And, you know, maybe that is the new
competitive advantage: collaboration,

not your competitive edge.

Scott: Mm-hmm.

Mm-hmm.

Yeah.

May Mo: Absolutely

Scott: Yep.

Okay.

Um, I, the, a whole new skill set to
be a part of this where you're working

on methodology, you're, you're working
on somebody else, uh, identifying the

problem, communicating the problem.

Uh, this, this is perhaps something
statisticians typically, you know,

we might be really good at solving
a problem if somebody hands it, but

there's a whole different skill set to,
to doing this, uh, uh, for statisticians

May?

May Mo: That's a question
or it's a statement?

Scott: well, you, you can say,
"I agree completely, Scott.

You're completely right."

I, I would be okay with that

May Mo: I, I completely agree with
that, and like a politician I might

Scott: Yeah, there you go

May Mo: uh, try to sell some
additional point to what you said.

Scott: There you go

May Mo: Um, really appreciate, um, one
thing is, um, helping us, um, to pushing--

to continue advocating us is there's
often not a single solution to problems.

And, um, our team the qu- and also
the broader bio stats function, the

quantitative teams, have the capability
to highlight the differences of those

options, be it it's a trial design,
be it it's analysis method, right?

The pros and cons.

And be able to show each one
assumptions we might base on.

Are those assumptions facts
or, or a guess, right?

And what's the risk if we assumed wrong?

And also, um, any kind of biases involved
in those design and analysis, right?

And what's the trade-off, uh,
probabilistic in terms of cost and

the future outcome and also patients?

so all those things I, I can see
is it become the new routine, and

I saw that a dawn for our group.

It's more roles, um, for us to, to, um,
more importance for us to work on those.

I have to say this is, um- like a
good news for, for both methodology or

innovation team and, uh, broader biostats.

And I, I saw encouraging, um, changes
in the industry, 'cause before just

the R&D head or the, the global
development organization start to

say, "I don't want team to come
propose one design, one design only."

I want to through differentiable and
credible so that leadership know how

those key questions were addressed, right?

Like the key thinking, what's
the risk of each of the design,

and what's the advantage of them?

encouraging me is saw recent communication
coming from our highest, um, program

governance, which is even R&D.

They said the same thing.

They want to see different
solutions, different options.

So-- And they want to see contrasting
those, the pros and cons, um,

potentially quantitatively.

So, so that's the type of opportunity
and s- this type of statist- uh,

statistical thinking the new coming
statisticians in, in the era of AI,

in the era a lot of new technologies.

I just wanna say that.

Scott: So I guess I'll, I'll change
the question and throw it to Casper,

is that feels like the ultimate r-
place that you'd want to be is, is

showing different choices to upper
management, whoever upper management is.

Uh, here are the different possibilities,
here's why I would do certain things, and

you're showing things that compel to them
that they can make the right decision.

I mean, that feels like the ultimate
place to be able to excel in, uh,

that you want to be if you're in
one of these methodology groups.

A challenge, of course, but
the ultimate place to be?

Kaspar: That's where

Scott: Yeah

Kaspar: are made, and that that's--
And, I mean, that's one thing.

Of course, that's an
attract-attractive place to be.

But the other thing is, it is my
genuine conviction that and quantitative

drug developers a lot to contribute
to those dis-- uh, decisions.

I mean, it's not like it's a good
place to be or so we wanna be there.

I think the point is have a lot to
contribute, as Mae was, was outlining,

kind of providing these options,
assign risks to the options, and then

boil it down and, and illustrate the
trade-offs decision-makers can make.

You can say, "Okay, if you change that a
little bit, it costs you ten million, but

then your probability of success increases
from sixty to sixty-seven percent.

one do you wanna buy?

It's your decision.

I provide those options."

And maybe, Scott, if I may, your initial
question was kind of maybe a little

bit what is the profile of a person?

And for me, the…

maybe the most important
aspect is curiosity.

Uh, a lot of what I have done just
started with a hallway conversation,

and somebody was just maybe in, in a--
bypassing, was just saying something.

And then I said, "Oh, wait a minute.

What do you mean?

Oh, is this something maybe we can help?"

Et cetera.

Or, "Is this something…"

Maybe then I get interested,
and I, I, I read a paper.

It's-- That curiosity is, I
think, what, what I see makes

some of these statisticians
influential in their company.

Kind of this wish understand the whole
thing, and then also this wish to kind

of be part of, of, of many of these
activities and to, uh, be able to

bring this quantitative thinking to

a relevant part of, um,
of the whole business.

And the other piece, so curiosity, and
the other big piece, in my opinion,

and we touched upon this already.

I mean, I assume the
technical background is

Scott: Right.

Right

Kaspar: of course.

But then the other thing, the
mentality piece, is this collaboration.

It, it's not like there is no hierarchy
between methods groups and project

statisticians and trial statisticians
and, and managerial statisticians.

We are all in this working together
to advance the molecules and every…

Like Mae very nicely outlined.

It's like doctors having different
specialties trying to cure the patient.

We are statisticians with different
specialties trying to advance

these molecules or kill them,
which may be even more important

Scott: Uh, uh, a huge part of this, and
I'll add to, I'll add to curiosity is

sort of a passion to, to, to, to answer
those questions that, you know, when I'm,

when I'm out running, I'm thinking about
how do I solve that problem, and I just,

I, I'm curious enough and passionate
enough, have the technical part of it,

but then be able to communicate it.

I mean, it's sort of the three
key things you've laid out in

the paper for a statistician.

Jenny, is that fair?

Jenny Devenport: Absolutely.

Um, I think, I think the curiosity piece
in particular really resonates with me

because we're often-- decision-makers
are often faced with, uh, with time

milestones to make a decision, uh,
where there's not a lot of clear data.

And so what can we do to
help enable those decisions?

And again, surface those biases and
surface those, uh, uncertainties,

and surface some of the options
that people don't go with their

gut, they go with the data.

And we're more creative also in what data
we can bring to inform those decisions.

Scott: All right.

So we, we,

Appreciate you all championing
innovation, moving statistics forward,

uh, better impact of statisticians.

I, I believe strongly that pharmaceutical
companies should ab- they invest in

R&D, they should invest in R&D within
statistics, development of statisticians.

I think they'll be better
pharmaceutical companies, and

appreciate all your work on that.

And thank you all out there in the
world of podcasts for listening.

And, uh, thank you for our listeners,
and thank you for joining us today.

And until next time, we'll
be here in the interim