Mr. Berry Goes to Washington

In this episode of "In the Interim…", Dr. Scott Berry marks the podcast’s one-year anniversary, sharing listener metrics, watch data, and regional engagement. He then delivers a step-by-step analysis of the FDA meeting process, detailing the progression from initial sponsor meeting requests and question submission to briefing book preparation, feedback cycles, and in-person logistics for a Type C meeting at the White Oak facility. Drawing from more than 25 years of trial design and regulatory experience, Scott offers precise guidance on technical preparation, sponsor responsibilities, and common errors in sponsor-FDA dialog, emphasizing what works and what wastes time inside the one-hour meeting constraint. His practical approach focuses on clarity, respect for process, and actionable advice.

Key Highlights

• Slightly over 30,000 people tuned in during the first year across 45 episodes; about 10,000 via audio, 20,000 via video with a global worldwide reach.
• FDA meeting workflow: request, submit four to eight questions, draft briefing book, receive written feedback, strict one-hour in-person discussion controlled by sponsor.
• Advice on briefing book content, avoiding new materials at the meeting, even what not to bring through the White Oak facility.
• Sponsor pitfalls: disingenuous patient advocacy, asking impossible questions, taking adversarial stance in statistical discussion.

For more, visit us at https://www.berryconsultants.com/

Creators and Guests

Host

Scott Berry

President and a Senior Statistical Scientist at Berry Consultants, LLC

What is In the Interim...?

A podcast on statistical science and clinical trials.

Explore the intricacies of Bayesian statistics and adaptive clinical trials. Uncover methods that push beyond conventional paradigms, ushering in data-driven insights that enhance trial outcomes while ensuring safety and efficacy. Join us as we dive into complex medical challenges and regulatory landscapes, offering innovative solutions tailored for pharma pioneers. Featuring expertise from industry leaders, each episode is crafted to provide clarity, foster debate, and challenge mainstream perspectives, ensuring you remain at the forefront of clinical trial excellence.

Judith: Welcome to Berry's In the
Interim podcast, where we explore the

cutting edge of innovative clinical
trial design for the pharmaceutical and

medical industries, and so much more.

Let's dive in.

welcome everybody to, in the
interim, I am your host, Scott Berry.

Appreciate you joining again.

I got a couple things I
wanted to talk about today.

Uh, I haven't done this before.

But one of the topics is going to be
in the interim, the podcast itself.

We are now, uh, a, a year into
this experiment, and I thought

I'd give you a little bit of, uh,
uh, of up, uh, update on that.

And then I'm gonna talk about, uh, so
today, when I, when I'm recording this, I

am, uh, a couple hours from jumping on a
plane and I'm heading to Washington, DC.

To visit with the FDA.

And I thought it might be nice just
there, there may many of you out

there, probably never been to the FDA.

What does this look like?

What does an FDA meeting look like?

Maybe some of you statisticians, maybe
you've done many of these and you'll

appreciate some of my stories of this.

So I, I, I, I'm a day away.

Uh, from that, um, and I thought I would
talk a little bit about going to the FDA.

So, um, uh, those will
be my two topics today.

I, I thought I would give you
some update on the podcast itself.

We, we here at Berry Consultant
started this podcast uh,

almost exactly a year ago.

Our first podcast, uh, was released unfit.

February 24th, 2025, and the story
of Barry consultants, uh, Don and

I both, uh, did that podcast and
we released 45 podcasts last year.

And I have to tell you, I'm, I'm
stunned by how many of you have come

up to me and say you really enjoy it?

Sent me email, uh, social media,
LinkedIn, different avenues.

Said they, you, you enjoyed it.

Whether it was an episode or.

Or coming back in.

I, I'm stunned by that.

I, I'm incredibly thankful, uh, of that.

I gotta tell you, I enjoy this,
uh, the, it's, it's, it's fun

for me to talk about this.

Uh, I, I've been, at this 25 years,
we've got tons of really talented people

at Barry doing really cool things.

And so it's, it's been really
fun to, to bring it to you.

But the, the reception has been great.

So.

For those of you who maybe join in
for multiple episodes, is anybody

out there watching sort of thing?

So we do have metrics on, is anybody
out there listening or watching?

We put out, uh, last year, 45 episodes.

We, we were essentially weekly, I
think we miss one week, uh, over

the holidays where we, we put
them out weekly, come out Monday.

Uh, starting from the end of
February 45 episodes over the year.

So this was, uh, so cut date
was the end of the year.

We had over 30,000, slightly over
30,000 people tuned into in the interim.

Again, I'm, I'm, I'm stunned by that.

And, um, uh.

Honored that you, you would join
in or, or find some value in this.

So, uh, I enjoy it.

I know the people at Barry that have
joined have, uh, have joined with me.

Kurt Veley, for example, has done
a ton of these as, as co-host,

and, uh, we've really enjoyed
about 30,000 people over that.

It's consumed through audio podcasts,
for example, it's on, uh, uh, apple.

Uh, podcasts, um, within that, um,
I'm not even saying that right,

but Apple Podcasts, for example,
you can listen to just audio.

By the way.

It's been one of the really interesting
things about this as somebody who's

been, who has used PowerPoint as
a crutch for so many years of.

Presenting as a statistician, you,
we do have a facility where I can

put slides and things in that.

I don't want to do that.

I want to be able to try to do this and
explain it or talk to the guests without

slides, and it's, it's, it, it really,
it's really hard to do, but I think it's,

it's great practice for me, but it's
great to really explain what it is without

being able to show you a graph, a formula.

In all of that, so many
people consume this by audio.

Only about 10,000 of those watches.

About a third were audio podcast
only, and, uh, two thirds were on

YouTube or Barry Consultant's website.

You can go to Barry consultant's
website and watch there.

Uh, I think that's just
YouTube, but about 20,000 people

joined on YouTube within that.

So, uh, uh, uh, more than
half of you consume this with.

The video tied to it, which is really
nice with the guests, um, to get their

impressions and, and all of that.

Um, so the various episodes,
what does that mean?

We do have watches per episode.

Now this is skewed towards early episodes.

They've been out longer, more opportunity.

So the, the very last one we did,
um, I, I guess the last one on my

databases was released in January.

It's on the, uh, path
to prevent prevention.

P two P trial for Parkinson's, by
the way, it's a fabulous podcast.

Uh, you should tune into that.

The very recent one has 222 watches.

It's only been out a couple weeks, and
it, I'm sure that's, that's been updated.

Now.

The, the most watched
episode of these was in June.

I did one on spending alpha.

Which I chuckle a little bit.

Uh, I, I'm a Bayesian and Alpha
doesn't mean much to a Bayesian.

We, by the way, we came out with
metrics within Barry and we do about

half of our trial, our frequentist
trials, and half of them are Bayesian.

The primary analysis
being Frequentist Bay.

Yeah, within that, so we
do a lot of frequentist.

We, we, we deal with Alpha, but talked
about spending alpha and the parts to it.

So that episode had 5,700 watches,
almost twice as big as any other episode.

And I think some people like it and a
whole new audience comes into all of that.

So I, it surprised me.

But, but somewhat interesting.

The second highest watched
episode was drug developers

listens, lessons from sports.

Specifically regression to the mean, and
that was 2,600 people have consumed that.

Uh, Nick joined me, Nick
Berry joined me on that.

And by the way, I owe you another
one on lessons from sports.

There's many of them out
there, uh, within it.

And that was, uh, 2,500 half of that.

The third was the story
of Barry consultants.

The fourth was seamless, two three trials.

I'm, I, I, I love that.

I was one of my, my, uh, uh, more.

Yeah, favorite EE episodes.

And then the fifth was the ARD
and Slog of innovating with

Mike Crams talking about that.

So, um, I surprise by some, one of
my favorite ones, for example, was

religion, politics, and ordinal
outcomes, which was the fifth

episode, and that has 372 watches.

So one 10th of it.

And I thought that was one of the
more enjoyable ones, uh, within it.

So I'm a little surprised at how
the numbers go, but it's, it's,

again, it is humbling and it is,
uh, thank you for joining in.

So if you've enjoyed this, uh, uh,
very much appreciated, we'll try

to continue with, uh, enjoyable.

Uh, content.

It, it has increased, uh, over
the time, not unexpectedly.

We had zero, uh, a year ago.

And, um, uh, so over time, somewhat
linearly increasing the number of them for

each, the number that consume it within
the first couple days, the, in the first

couple weeks, we do have global listeners.

About 37% of the watches
were the United States.

6% were Australia.

Uh, many colleagues and, uh,
uh, friends in Australia.

So appreciate that.

The UK is about 6%, India's
about 6% Germany, about 4%

Switzerland, uh, and down.

So, uh, Canada.

So really global representation, which
is, which is also very, very humbling.

And I guess one really interesting
thing, in December we were listed

on the top 10 popular shows.

For the mathematics
category of Apple Podcasts.

So we ended up number seven on the
math category for Apple Podcasts.

And so that was, that
was really neat to see.

So again, thank you on that and maybe
I'll come back and, and update you

on the various, uh, uh, metrics.

How are we doing here?

The, I, I've got some neat ones coming up.

I think some really neat
guests, uh, within that.

So today, what, what did I
want to talk about today?

It's really sort of front and
center of my mind that I'm

getting on an airplane today.

By the way, this is Austin, Texas
in the background, uh, here.

And I get on a plane in about
three hours and I'm headed to dc.

I have a meeting with the FDA tomorrow.

Uh, with a sponsor.

This is all a sponsor.

And I thought I'd give you
a bit of a, what is this?

What, what happens at these meetings?

What are they about?

Little bit of advice, uh, from it, and so.

This is a type C meeting, and
I'm not a regulatory expert.

I'm not gonna give you all
the, the, the differences of

type A, type B, type C meeting.

I think this is the most
general of the meetings where

you can go for general advice.

This, this one particular is proposing
a phase three trial for a, uh, for a

biologic actually, but, uh, for a drug.

And that's a common type meeting
that I, I experience is trying to

get approval for a development plan.

There are other meetings where you're
trying to get rid of a clinical hold.

You have specific issues, uh,
and those give different names.

But so focus on this 'cause it's, it's
typically the type of meeting I go to.

So I've, I've been doing this at Berry
Co Consultants slightly over 25 years.

Been to many of these meetings.

I, I don't count them.

I should count them.

It's more than a hundred,
uh, of these meetings.

They've been at the Center for
Biologics, center for Devices,

center for um, drugs within this.

So multiple types of meetings.

And so just a viewpoint of, of mine,
uh, on what these meetings look like.

Okay, and I'm gonna do this without
telling you anything about who the

sponsor is for the meeting tomorrow.

Any details of that?

Uh, as I, I, I can't
give you those details.

So just high level views, generic
aspects of these meetings.

So many of these meetings that
I'm involved in is we're, we're

proposing, uh, a particular design.

Now, these meetings can be hybrid.

It.

They can be written feedback only
from FDA or they can be in person.

This is an in-person meeting.

Interestingly, by the way, the
government might be shut down.

These meetings happen.

And, uh, there's a risk, uh, that the
government is shut down because sponsors

fund these, they come from the user
fees, and so they're paying for these.

And so these meetings happen.

They don't get canceled when
the government is shut down.

Other activities, uh, FDA people can't
travel to meetings and other activities,

but user fee funded things they have.

So there, there's no risk of that.

The typical scenario is you submit
a meeting request to the FDA.

We want to have a meeting with the FDA.

You can request in person, you can request
written feedback only, and then the FDA

decides what kind of meeting you get.

They can say, we're only going
to provide written feedback.

They can say, sure, we'll
do an in-person meeting.

They can do a hybrid meeting
where it's, it's all online.

Used to be telephone calls.

So I'm old enough that many of these were
done by telephone call only within that.

But now they are done through a a, a zoom
type teams type meeting as a possibility.

So they can say, no, we don't
want to do an in person meeting.

We're only giving you written feedback.

So you submit a request for a particular
day, timing of this within 60 days.

So a coup, you need to give them
a couple months notice, but you

also submit questions to the FDA.

So these meetings are not.

FDA grilling you.

You are asking questions of the
FDA, and I'll talk more about that.

But you list out the questions and
these are typically, there may be rules

to this, but these are typically four
to eight questions you ask of the FDA.

This is not 50 questions
to them, but four.

And yes, some questions
are, here's a question.

Here's part A, B, C, D, but
largely four to eight questions

that you ask of the FD, A.

And that goes with the meeting request.

And so they know who should put at the
meeting, the, the topic of it, decide what

kind of meeting it should be within that.

Um, and then within 30 days
you submit more detail to them.

So you submit what's called a
briefing book that has the sponsor

position on, on the questions.

So is this design appropriate for
a phase three trial for, uh, uh,

getting approval of this drug?

And that might be one
of the questions on it.

Uh, it might be an adaptive design.

Does the FDA agree that the adaptive
design controls type one error or

that the Bayesian borrowing or, or
whatever it is, uh, there may be a

question specifically about that.

Does FDA agree that this is an appropriate
way, uh, to do it and you put a position

you support why you're doing it?

You don't give, I generally,
everything I say here is generally

somebody's gonna say, Hey, I had
a meeting where X, Y, Z happened.

I'm sure, I'm sure.

So these are very general comments,
but you, you don't throw, you don't

submit five designs and say, FDA,
what, which one do you like better?

You submit your proposed design
and you support it, and you, you

present why you support that design.

Within it, uh, that's an
important part of this.

You're asking them those questions
around it and you're supporting it.

The, the, so writing the briefing
book is a big part of it.

Uh, submitting a a synopsis
is part of this submitting.

You don't submit a statistical
analysis plan this early, but you

give the details needed to support
the various aspects of the design.

And I'm gonna focus on the
statistical parts of this.

And, and you shouldn't be
surprised by, by that part of it.

Now, high level is what happens
is, so we have a meeting.

Tomorrow, uh, it's a Wednesday,
and we got feedback from the FDA.

The first contact about their responses
to them don't happen at the meeting.

It.

They provide written feedback.

We got that Friday afternoon,
and I don't know the rules.

It's like within 72 hours or 48 hours
or something that they're required to

respond in writing about your questions.

And so we got those late Friday
afternoon and um, uh, within that,

and you get that feedback now.

I'll spend a little bit of time on what
kind of happens, uh, in the sponsor.

And it typically happens
once that feedback gets in.

The one of the reasons I, I think,
for example, they have, they may

have written up all those responses
and had them two weeks ago.

But they don't submit them, and I
think the reason why they wait till

kind of to the end is they don't want
the sponsor creating new material

for them to have to evaluate.

It.

And so the meeting, you can't
bring new material to the meeting.

Everything you submitted, the briefing
book is fair game for questions for that.

You can ask them to clarify things
about their position on it, but

you can't say, Hey, we went back
and we found this new paper.

Um, we, we, we created this new design.

What do you think of this?

They haven't had time to review it.

They, they won't answer those
questions in the meeting.

No new material.

And I think that's why the
written feedback comes so late.

Sure.

Sponsors would've it like to have it
two weeks earlier, uh, to, to really

plan and, and scenario around that.

So that comes relatively late.

And I think there's a purpose to that.

These meeting, this meeting's one
hour, and I think most of these, every

once in a while, there's a, there's a.

Hour and a half meeting, but I
think the typical meeting is one

hour and it is strictly one hour.

So I've been in a number of these
meetings where I think the sponsor

largely wastes time and I'll
come back to that a little bit.

And when that hour's up, it's over.

The project manager takes
you and you leave the meeting

or the zoom meeting ends.

It's over.

You did.

You wanted to spend 20 more
minutes telling them how great

your drug was and it's over.

That's your shot.

So it's one hour is really, really short.

And so planning that what you're
gonna do in that appropriately.

The other part of this is, this
is your meeting as the sponsor.

This is not the FDA's meeting.

They don't bring the agenda where they
want to talk more about X, Y, or Z.

They show up and you submit an
agenda and you get to ask the

questions as the sponsor, uh,
within that, and it's your meeting.

If you think their responses
that they supplied to the first

three questions are sufficient.

You don't even talk about
those in the meeting.

You don't need to go through them.

And, but, but question four,
that's the, that, that's really,

what you want clarification, you
want further discussion on that?

That's open game and you
might submit to them.

So then largely you put an
agenda together, you, you

you submit back to the FDA.

We would like to talk about
question four and question seven.

Those are the topics.

So the FDA knows sort of what's gonna
happen at the meeting within that.

And again, they.

They don't want new material because
they don't have time to review

it with within that timeframe.

Um, you can also cancel the meeting
at that point, the written response.

If you got a yes to every question,
your design looks great, you.

Don't need the meeting.

And we've had those, it, it's not
that uncommon, actually, I don't know,

25% of meetings might be canceled,
that the response is sufficient.

There's no clarifications
needed within that.

And, and that's, that's there.

I I think generally sponsors don't like
to cancel the meeting anytime they get in

front of FDA, that they want to do that.

And, uh, but.

Um, that there are times when
absolutely we've got our responses.

We know what to do.

We don't, we don't have
any additional questions.

Sometimes my part of the meeting
is canceled, so the design, the

statistical part, they want to talk
about manufacturing or they want to

talk about a particular toxicology
test, and that that's the only thing,

the question on that, that they have.

And so they, they, they change
who's going and, and I, and

I might not go within that.

I am going to the meeting
tomorrow, for example.

Okay.

So that's typically, uh, uh, kind
of what these meetings look like.

So sort of high level, um, uh,
the, what does the prep look like?

So once you get that response, it's, it's
reading the responses, it's consuming it.

There's, these are worded
incredibly precisely, and

the reason why I say that is.

There's, um, there are different
kinds of no in it and there are

different kinds of advice in it.

The FDA can say, absolutely no, we will
not allow you to do that to patients.

And it's a hard no.

You's also softer nos, and there's,
you can do that and it's your risk.

We suggest you don't do
that and you do X, Y, and z.

There's also, um, largely this sort
of feedback that we think you should

do this sort of other thing, but.

They're not saying no kind of thing.

So it's consuming all of this
and largely what do we get about

this feedback and deciding do we
need, do we need clarification?

What do we need?

What are the parts of this?

And this is the real important part
and it's, it's kind of hectic because

you need to get back to the FDA.

They needed to submit.

Yesterday morning, I think, uh,
what they want to do and any

new, any slides they wanna show
to the FDA had to bid yesterday.

So they need to really decide,
do we have the meeting?

What do we want to talk about?

And a lot of that is, um,
what, what's a win here?

What are we trying to accomplish by that?

And it, by having the
meeting, so there's this huge.

Frequent calls.

Lots of discussion once
that thing comes in.

And of course it was over the
weekend this time, uh, having these

various comments to it and really
deciding what are we gonna do?

What are the things that
we want to talk about?

Get clarification, and then really
identifying what's a win here?

Why, why do we need clarification?

What do we want to have happen?

What's a good outcome?

What's a bad outcome?

These parts all go into to,
to then decide, and you, okay,

here's where we're going with it.

Um, within that, so that this kind
of all happens hurriedly and that's

for me about to get on a plane.

That's all happened.

We've had these discussions
within that it, it typically, um.

Typically then you, you
meet with the sponsor.

And by the way, I've done
this with academic sponsors.

I've done this with patient
organizations being the sponsor

who submitted with pharmaceutical
companies, a medical device company.

So sponsors come in many, many
flavors within this, typically

we, you have pre-meetings to it.

There's some weird aspects to
it that I'm not actually there

today within it, but we are.

Tomorrow morning, we
have multiple meetings.

We're gonna have, you know, four
or five hours where we're gonna go

through what that all looks like.

So we'll, we'll, we'll
meet tomorrow morning.

Within that, we, we largely, for this
one, there's only a limited set of things

we needed to discuss and so we, we.

Didn't necessarily need to show up a
full day in advance, but a lot of times,

this might even be two days in advance,
you're planning out this one hour meeting

and what do we want to talk about and
how we're gonna do it, and all of that.

Um, and then typically, you all,
you, you're all at a meeting place.

You get on a big bus and you drive to
the white Oak facility in Washington, DC.

Yes.

Suit and tie.

Uh, within that I did have a sponsor once.

Tell me, and I'll chuckle at this,
uh, not to wear your expensive Italian

suit, and I chuckle because I'm a
statistician and I own one suit.

Um, and it's not an expensive Italian
suit, uh, within it, by the way.

I have multiple kids getting married and
all that, so I did buy a tuxedo, and I

don't wear a tuxedo to the FDA, but I do
own a tuxedo for the various weddings.

Uh, uh, I have lots of cousins and I,
nephews and nieces and lots of weddings,

and so it's, it's a nice thing to have,
but I, I own one suit, so, uh, I was at

least given that, and I think they're
worried about, um, and I have no idea,

but, but that, uh, uh, doctors show
up, high paid doctors show up to a

place, and, and FDA makes less money.

Uh, within this, they then, if you got a
position at a sponsor, for example, um,

you go to the White Oak facility.

Building 22 is where the Cedar
and Seber meetings happen.

All the, the, the, the
conference rooms there for it.

And building 66 is where
you go to the CDRH meetings.

That's sort of down the hill
a little bit from there.

Uh, you drive into this facility.

Uh, it's interesting, I
predate this facility where.

The, the meetings took place at
sort of sporadic office buildings

before the White Oak facility.

The White Oak facility's awesome.

It'd be great if it was
closer to the Metro.

Um, I, I think they actually have buses
for employees there, but, uh, within

it, it's now very highly secured.

And so it's, it's, there's
a wide fence to this.

You go through, uh, uh, security
comes out, they're looking under your

car, they're checking everything.

So it's a very, very secure building.

Once you get in and then you drive
to the building, and then further

you go in there and you go through
additional security at that point, um.

And we, I interestingly, they, they,
they can be almost, I, it's changed

a little bit, but four or five years
ago, they were almost, I'll say, more

thorough than the airport, and they
asked people to pull every single

cord out of their bags and all this.

So you'd have these clinicians would
show up with this big carry bag, and

they're pulling everything out of it.

Extremely slow within this.

Everybody's checked in.

IDs are checked.

Foreign visitors,
passports are gone through.

So, uh, hit number one.

Don't bring anything to the meeting.

I don't bring anything.

I leave everything on the bus.

Uh, I bring a, a pad
of paper and a pencil.

A a, a pen or a pencil and I don't bring
my computer, I don't bring any coin cords.

I generally have my phone to
keep track of time within that.

Really easy to go through security with.

Within that, um, by the way, when
you go into these facilities,

they have big pictures of the
president of the United States.

The head of Health and Human Services
and the commissioner of the FDA, maybe

the Deputy commissioner of the FDA,
but these big prominent pictures,

when you go in, uh, of that, um,

then right on time, project
manager takes you to your meeting.

It starts right on time.

And again, that meeting's yours,
you can't record the meeting.

Uh, within that, uh, it is very explicit.

There's no recording of it.

Minutes are taken within that, and
the minutes are always reviewed

and they, they, they create
official minutes of the meeting.

But no, no recordings of it are done.

And the FDA team is typically there.

You go in right out there at 1 0 1
and uh, one o'clock, the FDA team

is there and it's your meeting.

And there might be introductions.

Typically in person.

There's introductions of the people,
uh, hybrid meetings typically you, you

introduce yourself and you start talking.

'cause there might be 40
people on this part of it.

And the discussion happens
and it's your meeting.

You can ask questions, you
can present within that.

And I'll talk more, uh, about that.

The other part is the meeting ends
at one hour, and I've been in those

situations where essentially the FDA
walks out and the project manager takes

you out and you, you again, you, you
want to say more and you can't now.

I, I do, I I have seen cases where hall
discussion happens and the FDA wants to

make sure they tell 'em X, Y, or Z, but
you can't count on that and it's probably

shouldn't be happening, sort of thing.

But they also have the room scheduled,
so they are going to get you out.

And so using that one hour
appropriately, it's huge.

Uh, within it, you're escorted out.

And then typically after the meeting,
you might go back to the hotel, discuss

what they said, where should we go?

What should we do, what
work needs to be done?

Typically, there's a rush to the
airport, uh, within it, a bunch of

people headed off to the airports.

You try to get out, people get back,
uh, within that, and that's kind of

the day, and that's kind of what my, my
day's going to look like within that.

I, I wanna be clear.

So, so suggestions for these
meetings within that, and I've

seen these meetings go very well.

I've seen these, me
meetings go quite poorly.

Um, within it.

I've probably done a
bad job in some of them.

Uh, within that.

The FDA is incredibly smart.

The, by the way, the FDA is really good.

And these are people that are
doing this for the right reason.

They're dedicated.

They want to see safe and effective
treatments go to patients.

There's nobody there that is just a hard
ass and my job's to make it hard on you.

It's not that at all.

They want you to be
successful a hundred percent.

It's good for patients.

They want you to be successful.

Within that.

And so that's a big part of it.

They're on your side now.

You can do things that make it, they're
not on your side anymore, and you

build up walls and you create things.

For example, starting the meeting where
you're about to run a phase three trial,

reminding them how it's so utterly
clear how good your drug is in it.

You, you don't know why the
FDA hasn't approved it yet now

means they're not on your side.

If, and, and there are circumstances where
maybe you've run one phase three trial,

you think the drug should be approved,
whatever, and you, you disagree with

that FDA meeting that's creating a fence
between them and there are lots of ways in

which these fences are created within it.

Now they're not on your side because
you're forcing them to tell you again, no.

And, and now there's this
sort of wall aspect of it.

But, but again, they all want safe and
effective treatments to be out there.

We're all patients, we're all humans,
and they're incredibly good at this.

Now the other thing that I see that
I struggle with, you have a one hour

meeting and way too much time is
spent and people can disagree with me.

Telling the FDA, how bad the disease
is, how much patients need a treatment.

They know this.

They absolutely know this, and
the fact that patients need

something doesn't mean that we're
now gonna approve your treatment.

They don't need bad treatments.

They don't need ineffective
treatments, non-safe treatments.

It's largely a waste of time.

And I'll sit there and this will go
on for 10 or 15 minutes where this

is done and there's no point to it.

Um, and it's actually, I think it's a
little insulting that they think that

you need to go through those details
and remind them of, of how bad this

disease is, for example, and how many
patients need this and, and that now.

Obviously to back up there,
there are strange diseases.

There are strange scenarios where it's
appropriate to review the information

out there and the FDA might not have an
expert in this situation, by the way.

They do bring in experts, they bring
in external people that are working

for the FDA that know this particular
disease, this subset, that sort of thing.

So, you know, generally there,
there is that, but if you're

going in there in a relatively.

Known disease, even a rare disease,
you're going in there and a LS and saying,

my meeting's not for a LS, by the way,
and saying patients need something they

know a LS, and I think it's insulting.

So, and again, what's the point to it?

You're not moving the ball forward.

You have one hour to try to figure out.

So spend a lot of time
figuring out what success is.

You know, what do you
want out of this meeting?

Um, uh, within that and then be very,
sort of driven by that particular goal.

And I'll present examples where I, I
probably didn't do very well in that.

Uh, within it.

Another way to prevent this sort of,
they're not on your side anymore, is

when you ask them impossible questions.

It drives, drives me nuts when
you ask them impossible questions.

If we see success in this
trial, will you approve us?

It's impossible Question.

Why ask that question.

You're gonna get, they're
gonna say it's a review issue.

We'll gather the information.

It's a review issue.

It's not a productive
question within that.

And you know the answer they're
gonna give you think about how would

you respond to these questions?

And many of 'em, I don't know
how to respond to that, you

know, but it's not productive.

It doesn't move the ball
forward as part of that.

Now statistics in in particular
there, it's not infrequently that.

They've, uh, presented feedback
on the design within that.

Um, I think maybe we've gotten it
wrong a little bit in, in, in that now,

to some extent, I think they've said
something wrong, but that's our fault.

We've provided all the material and all
of that, and so there are cases where

we wanna walk through the details.

We didn't explain this very well, and the
reason why is you'd really like to get

this part of the design approved, whether
it's an adaptation or that part of it, and

you think that further discussion would
clarify the particular issue, uh, on it.

And that's really the goal of
that discussion in that part

of the, the, the presentation.

And that may be a part where I'm
speaking about specifically about that.

And, and in my mind, by the way,
I may think they said something

wrong, but I can never say that
and I, I, I'll say more about it.

So the point may be sort of stat related
in that now the, um, the important

part if I'm in a scenario like that
is you've got to give the FDA an

out and being right might be wrong.

You know, stomping on somebody and saying,
you're stupid for not knowing this.

This is obvious, this is this, this.

What do you think you're going to get for
a response from them for, for a scenario?

So let me give you an
example, years ago example.

Uh, it's not part of.

My discussion tomorrow, we, it was a
relatively bad disease, wasn't oncology,

but feels like oncology a little
bit, where, um, mortality is a risk.

Uh, it's not mortality's, not just a risk.

This isn't intensive care where they
die within 30 days or they live,

this is relatively a deadly disease.

High risk of mortality, maybe
median survival, six to nine months.

And, uh, in this scenario, a bad disease.

We proposed an analysis at a randomized
trial with a Cox proportional hazard

model as the primary analysis.

The FDA in their written feedback said no.

And again.

What kind of, no.

Sometimes they say you can do that, but
we'd prefer X and it's kind of, alright,

do we do something different in this?

But they said, we want
a landmark analysis.

What that means is, and I think they said
six months, we wanna know the proportion

of patients that are alive at six months

now.

I think that's a bad analysis in this.

Um.

And it's inappropriate.

By the way, oncology trials have this
constant risk of mortality and all that.

They don't do landmark
analysis at one year.

They summarize that, but that's not
the primary analysis of these trials.

So the huge part is I want to get
them to accept a Cox model, a hazard

ratio, instead of a landmark analysis.

Now.

The Cox model is gonna be more
powerful, but I think it's

more clinically appropriate.

It's more appropriate for the drug.

And so now I'm presenting this and if
I come across as this is just stupid

oncology's figured this out, you know, the
person can largely on the other end draw

this and say, you know, I'm the regulator.

And you've, you've sort of
alienated them at this point.

You haven't given them an
out, and I'm the regulator.

This is the decision.

And you're kind of done at that point.

So instead, you can walk through,
show the differences on, and, and, um,

acknowledge why the person did that.

And largely I think they do that
because it's a really nice summary.

The knowing that 70% of people
are alive at six months relative

to 40% alive at six months is a
pretty easy thing to understand.

The Sta, there aren't a lot of
statistical assumptions to that.

Very straightforward.

Now, the challenge here
is, is it six months?

Is it nine months?

Is it 12 months?

And so you can sort of walk through
this and what we, we entertain

doing these landmark analysis.

We think we, we may even push
patients in the right time to do it.

Might be 12 months, it
might be six months.

The Cox models really
integrating all of these.

So you've, you've represented to them.

You understand why they
came back with this comment.

Here's why we'd like to do it.

By the way, we're gonna show you
the six month rate and that'll be a

secondary endpoint and all of that.

Do you think that's an
appropriate way forward?

So you've, you've, you've
respected what they did.

You've acknowledged why they say that.

You've presented why you think your
way is better in a way that they

can say yes to that and it's not.

You were right and I was wrong.

You don't want a yes to be an
acknowledgement that they were

wrong and you were right, because
then you're not gonna get a yes.

And that's a lot of times
what I'm thinking about.

And the, the challenging part of
this, uh, uh, is it, and it's always

sort of the challenging part of that.

So allowing a yes to be had in that.

And sometimes it's not even a yes.

It's not a no.

Uh.

I under and, and if they say, I
understand your argument, we're gonna be

particularly interested in six months.

We think you should do six months,
but you can do a Cox mom, that

might be a great outcome for you.

And it's not a no, and,
and and all of that.

So again, that's the part
of this, um, uh, within it.

And there are some times where, you
know, you're never gonna get them to

say yes, but you're trying to get them
not to say no in a particular scenario,

uh, uh, for the particular design.

You gotta give 'em that out in a way
that is, is, uh, allows respect and,

and, and recognizing, uh, of that.

Um, and that can be a challenge for me.

Sometimes what gets written in there
is something statistically that

I think is wrong and I, I wanna
win that battle and, you know,

that's not the point to it, to it.

Um, you know, I can be wrong by trying
to show that I'm right in that scenario.

It, that's how productive
it doesn't move the ball.

Written feedback is
much easier to provide.

Something is not a game changer.

It doesn't alter where you're going,
but you think they haven't understood

why your information fraction
for a group sequential design is

appropriate or, or, or something else.

It written feedback.

Is enough for that and it's easier to
accept things or research things or wi wi

within that, but it might not be something
critical to where it goes within that.

So written feedback as a
statistician to things you think

are wrong can be much better than
a discussion in front of everybody.

Many times I'm presenting
statistical stuff and I'm trying

to appeal to the clinical team
as much as I am the statistician.

If this is a scenario of Bayesian
borrowing of an alternative data

set, part of it is the mechanism,
the Bayesian hierarchical model that

I'm presenting, but I'm really trying
to appeal to the clinician why this

is clinically appropriate, how it
behaves, what the estimate's gonna

look like, what's the confidence
interval in a way that they see it.

And, and that even though I'm
the statistician, I'm kind of

presenting to somebody else is,
is a challenge that I'm frequently

dealing with, uh, in these scenarios.

Um, I, I had an experience one time
with Roger Lewis who works here at

Berry Consultants, a brilliant, uh, um.

Clinician, he's an emergency room
doctor, but a, a brilliant guy.

Um, I was sitting next to him at one of
these FDA meetings and somebody from the

FDA said something that I thought was just
wrong, and I thought we had a really good

argument and I was just so excited to jump
up and say, and, and I started to respond.

Meanwhile, Bob Temple.

Responded to the FDA person
and was making my point.

Roger grabbed me and largely
really looked at me like, really,

Bob Temple is making your point.

Shut up.

You know, kind of thing.

So, uh, and I did, I I took the hint,
I shut up and it's much better coming

from, I didn't have to say anything.

The point was made.

It's very different when somebody
from the FDA presents that part of it,

and not, and not me, uh, within that.

But, so I can sometimes get
excited about trying to be right.

And remember, it's, it's all
part of a bigger battle, uh,

of what we're trying to do.

Um, uh, within that, I don't get,
I, i i a last resort for me is

appealing to guidances appealing to
a paper from a famous statistician

that says, A is better than B.

Within that.

I, I, you don't get very far with
that, and I feel it's almost a

last resort to say the own, your
own FDA guidance says X, Y, Z.

It's almost bringing legal standards
into it, and at that point it's somewhat.

I, I, I don't think it's productive now.

There, there can be time and place
for it, and I know that can be part

of a written response as we think
this is also supported by this, but

trying to use that to win an argument,
I, I, I've just never been very

successful, uh, at that part of it.

Now I kind of live.

In many different spaces
than, than others.

And it depends on what you're discussing
within that, you know, if it's a guidance

on secondary endpoints and the way
something is done, and it's relatively,

it's a relatively non, uh, critical part.

Yeah.

Uh, absolutely.

We're, you know, we, we tried to follow
this guidance sort of thing, but to

prove a point, pulling out guidances
haven't been successful at me, so I.

I try never to do that.

I feel like once you break out the legal
stuff, this is the, the productivity

of the meeting is kind of shot.

Okay.

So at the end of the meeting, the
hour is up and you're, you're.

Project manager from the FDA reminds
anything we're expecting X from the this.

You'll hear this and it's usually
we're gonna get the minutes finalized,

you know this, but very, uh, uh,
sort of process part of this.

And they take you out and the meeting's
over, uh, and typically everybody runs to

the airport at that point, or you may have
a, a post meeting within a, any of that.

So.

That'll be my day tomorrow.

Uh, within this, I'm very
much looking forward to it.

I, I always look forward
to these meetings.

I, I think it's really,
uh, having an impact.

So I, I, I love these meetings and again,
I, I, I, because I truly respect what the,

the people at the FDA are doing and their
role they play in, in all of this process.

So, uh, this will be exciting.

So thanks everybody for joining me.

Uh, wish me luck tomorrow.

Wish science luck tomorrow.

Uh, and until next time, we'll
be right here in the interim,