WEBVTT

NOTE
This file was generated by Descript 

00:00:00.000 --> 00:00:03.410
Judith: Welcome to Berry's In the
Interim podcast, where we explore the

00:00:03.410 --> 00:00:07.710
cutting edge of innovative clinical
trial design for the pharmaceutical and

00:00:07.710 --> 00:00:10.090
medical industries, and so much more.

00:00:10.279 --> 00:00:10.979
Let's dive in.

00:00:12.000 --> 00:00:16.860
welcome everybody to, in the
interim, I am your host, Scott Berry.

00:00:17.580 --> 00:00:19.110
Appreciate you joining again.

00:00:19.110 --> 00:00:21.450
I got a couple things I
wanted to talk about today.

00:00:21.480 --> 00:00:23.340
Uh, I haven't done this before.

00:00:24.435 --> 00:00:29.535
But one of the topics is going to be
in the interim, the podcast itself.

00:00:29.835 --> 00:00:34.815
We are now, uh, a, a year into
this experiment, and I thought

00:00:34.845 --> 00:00:38.835
I'd give you a little bit of, uh,
uh, of up, uh, update on that.

00:00:39.225 --> 00:00:45.735
And then I'm gonna talk about, uh, so
today, when I, when I'm recording this, I

00:00:45.735 --> 00:00:52.035
am, uh, a couple hours from jumping on a
plane and I'm heading to Washington, DC.

00:00:53.100 --> 00:00:54.955
To visit with the FDA.

00:00:56.190 --> 00:01:00.360
And I thought it might be nice just
there, there may many of you out

00:01:00.360 --> 00:01:02.910
there, probably never been to the FDA.

00:01:02.940 --> 00:01:04.350
What does this look like?

00:01:04.620 --> 00:01:06.960
What does an FDA meeting look like?

00:01:07.170 --> 00:01:10.560
Maybe some of you statisticians, maybe
you've done many of these and you'll

00:01:10.560 --> 00:01:12.570
appreciate some of my stories of this.

00:01:12.750 --> 00:01:15.720
So I, I, I, I'm a day away.

00:01:15.945 --> 00:01:20.805
Uh, from that, um, and I thought I would
talk a little bit about going to the FDA.

00:01:20.835 --> 00:01:24.255
So, um, uh, those will
be my two topics today.

00:01:24.360 --> 00:01:27.765
I, I thought I would give you
some update on the podcast itself.

00:01:28.575 --> 00:01:31.995
We, we here at Berry Consultant
started this podcast uh,

00:01:31.995 --> 00:01:33.435
almost exactly a year ago.

00:01:33.435 --> 00:01:36.975
Our first podcast, uh, was released unfit.

00:01:37.575 --> 00:01:46.155
February 24th, 2025, and the story
of Barry consultants, uh, Don and

00:01:46.155 --> 00:01:50.955
I both, uh, did that podcast and
we released 45 podcasts last year.

00:01:51.555 --> 00:01:55.725
And I have to tell you, I'm, I'm
stunned by how many of you have come

00:01:55.725 --> 00:01:57.795
up to me and say you really enjoy it?

00:01:57.795 --> 00:02:02.175
Sent me email, uh, social media,
LinkedIn, different avenues.

00:02:02.175 --> 00:02:03.675
Said they, you, you enjoyed it.

00:02:03.675 --> 00:02:04.965
Whether it was an episode or.

00:02:05.190 --> 00:02:06.420
Or coming back in.

00:02:06.600 --> 00:02:08.070
I, I'm stunned by that.

00:02:08.070 --> 00:02:10.740
I, I'm incredibly thankful, uh, of that.

00:02:10.740 --> 00:02:15.150
I gotta tell you, I enjoy this,
uh, the, it's, it's, it's fun

00:02:15.150 --> 00:02:16.800
for me to talk about this.

00:02:17.070 --> 00:02:22.020
Uh, I, I've been, at this 25 years,
we've got tons of really talented people

00:02:22.020 --> 00:02:24.240
at Barry doing really cool things.

00:02:24.630 --> 00:02:27.540
And so it's, it's been really
fun to, to bring it to you.

00:02:27.540 --> 00:02:29.310
But the, the reception has been great.

00:02:29.310 --> 00:02:29.610
So.

00:02:30.315 --> 00:02:34.695
For those of you who maybe join in
for multiple episodes, is anybody

00:02:34.695 --> 00:02:36.945
out there watching sort of thing?

00:02:37.125 --> 00:02:41.865
So we do have metrics on, is anybody
out there listening or watching?

00:02:42.675 --> 00:02:47.115
We put out, uh, last year, 45 episodes.

00:02:47.145 --> 00:02:52.635
We, we were essentially weekly, I
think we miss one week, uh, over

00:02:52.635 --> 00:02:57.645
the holidays where we, we put
them out weekly, come out Monday.

00:02:58.065 --> 00:03:03.285
Uh, starting from the end of
February 45 episodes over the year.

00:03:03.345 --> 00:03:07.005
So this was, uh, so cut date
was the end of the year.

00:03:07.335 --> 00:03:14.205
We had over 30,000, slightly over
30,000 people tuned into in the interim.

00:03:15.000 --> 00:03:17.625
Again, I'm, I'm, I'm stunned by that.

00:03:17.955 --> 00:03:19.305
And, um, uh.

00:03:20.385 --> 00:03:24.765
Honored that you, you would join
in or, or find some value in this.

00:03:24.765 --> 00:03:26.415
So, uh, I enjoy it.

00:03:26.415 --> 00:03:30.615
I know the people at Barry that have
joined have, uh, have joined with me.

00:03:31.125 --> 00:03:35.715
Kurt Veley, for example, has done
a ton of these as, as co-host,

00:03:36.045 --> 00:03:41.355
and, uh, we've really enjoyed
about 30,000 people over that.

00:03:41.355 --> 00:03:49.245
It's consumed through audio podcasts,
for example, it's on, uh, uh, apple.

00:03:49.590 --> 00:03:54.960
Uh, podcasts, um, within that, um,
I'm not even saying that right,

00:03:54.960 --> 00:03:58.290
but Apple Podcasts, for example,
you can listen to just audio.

00:03:58.620 --> 00:03:58.980
By the way.

00:03:58.980 --> 00:04:02.490
It's been one of the really interesting
things about this as somebody who's

00:04:02.490 --> 00:04:07.260
been, who has used PowerPoint as
a crutch for so many years of.

00:04:07.375 --> 00:04:12.115
Presenting as a statistician, you,
we do have a facility where I can

00:04:12.115 --> 00:04:13.855
put slides and things in that.

00:04:13.855 --> 00:04:15.175
I don't want to do that.

00:04:15.595 --> 00:04:20.605
I want to be able to try to do this and
explain it or talk to the guests without

00:04:20.605 --> 00:04:27.415
slides, and it's, it's, it, it really,
it's really hard to do, but I think it's,

00:04:27.415 --> 00:04:32.275
it's great practice for me, but it's
great to really explain what it is without

00:04:32.275 --> 00:04:34.525
being able to show you a graph, a formula.

00:04:34.910 --> 00:04:38.990
In all of that, so many
people consume this by audio.

00:04:38.990 --> 00:04:41.300
Only about 10,000 of those watches.

00:04:41.300 --> 00:04:47.900
About a third were audio podcast
only, and, uh, two thirds were on

00:04:47.900 --> 00:04:50.510
YouTube or Barry Consultant's website.

00:04:50.510 --> 00:04:52.730
You can go to Barry consultant's
website and watch there.

00:04:53.030 --> 00:04:57.110
Uh, I think that's just
YouTube, but about 20,000 people

00:04:57.110 --> 00:04:59.090
joined on YouTube within that.

00:04:59.090 --> 00:05:03.560
So, uh, uh, uh, more than
half of you consume this with.

00:05:04.095 --> 00:05:08.475
The video tied to it, which is really
nice with the guests, um, to get their

00:05:08.475 --> 00:05:10.305
impressions and, and all of that.

00:05:11.250 --> 00:05:15.180
Um, so the various episodes,
what does that mean?

00:05:15.390 --> 00:05:18.330
We do have watches per episode.

00:05:18.330 --> 00:05:21.750
Now this is skewed towards early episodes.

00:05:21.750 --> 00:05:23.910
They've been out longer, more opportunity.

00:05:24.240 --> 00:05:29.790
So the, the very last one we did,
um, I, I guess the last one on my

00:05:29.790 --> 00:05:33.030
databases was released in January.

00:05:33.060 --> 00:05:36.390
It's on the, uh, path
to prevent prevention.

00:05:36.975 --> 00:05:40.995
P two P trial for Parkinson's, by
the way, it's a fabulous podcast.

00:05:41.325 --> 00:05:42.585
Uh, you should tune into that.

00:05:42.645 --> 00:05:45.465
The very recent one has 222 watches.

00:05:45.585 --> 00:05:49.185
It's only been out a couple weeks, and
it, I'm sure that's, that's been updated.

00:05:49.185 --> 00:05:49.455
Now.

00:05:50.205 --> 00:05:57.525
The, the most watched
episode of these was in June.

00:05:57.975 --> 00:06:00.100
I did one on spending alpha.

00:06:01.065 --> 00:06:02.745
Which I chuckle a little bit.

00:06:02.805 --> 00:06:07.875
Uh, I, I'm a Bayesian and Alpha
doesn't mean much to a Bayesian.

00:06:08.445 --> 00:06:12.135
We, by the way, we came out with
metrics within Barry and we do about

00:06:12.135 --> 00:06:16.035
half of our trial, our frequentist
trials, and half of them are Bayesian.

00:06:16.035 --> 00:06:18.850
The primary analysis
being Frequentist Bay.

00:06:19.260 --> 00:06:22.590
Yeah, within that, so we
do a lot of frequentist.

00:06:22.950 --> 00:06:27.750
We, we, we deal with Alpha, but talked
about spending alpha and the parts to it.

00:06:27.960 --> 00:06:34.590
So that episode had 5,700 watches,
almost twice as big as any other episode.

00:06:35.130 --> 00:06:39.720
And I think some people like it and a
whole new audience comes into all of that.

00:06:39.720 --> 00:06:41.430
So I, it surprised me.

00:06:41.955 --> 00:06:43.455
But, but somewhat interesting.

00:06:43.455 --> 00:06:47.715
The second highest watched
episode was drug developers

00:06:47.715 --> 00:06:49.815
listens, lessons from sports.

00:06:51.150 --> 00:06:57.240
Specifically regression to the mean, and
that was 2,600 people have consumed that.

00:06:57.570 --> 00:07:00.840
Uh, Nick joined me, Nick
Berry joined me on that.

00:07:01.080 --> 00:07:05.640
And by the way, I owe you another
one on lessons from sports.

00:07:05.640 --> 00:07:08.790
There's many of them out
there, uh, within it.

00:07:08.790 --> 00:07:11.940
And that was, uh, 2,500 half of that.

00:07:12.390 --> 00:07:15.480
The third was the story
of Barry consultants.

00:07:15.990 --> 00:07:19.050
The fourth was seamless, two three trials.

00:07:19.050 --> 00:07:20.640
I'm, I, I, I love that.

00:07:20.640 --> 00:07:23.865
I was one of my, my, uh, uh, more.

00:07:25.200 --> 00:07:28.050
Yeah, favorite EE episodes.

00:07:28.260 --> 00:07:32.310
And then the fifth was the ARD
and Slog of innovating with

00:07:32.310 --> 00:07:35.010
Mike Crams talking about that.

00:07:35.700 --> 00:07:42.720
So, um, I surprise by some, one of
my favorite ones, for example, was

00:07:42.750 --> 00:07:46.170
religion, politics, and ordinal
outcomes, which was the fifth

00:07:46.170 --> 00:07:50.100
episode, and that has 372 watches.

00:07:50.805 --> 00:07:52.395
So one 10th of it.

00:07:52.395 --> 00:07:56.355
And I thought that was one of the
more enjoyable ones, uh, within it.

00:07:56.355 --> 00:07:59.325
So I'm a little surprised at how
the numbers go, but it's, it's,

00:07:59.445 --> 00:08:04.335
again, it is humbling and it is,
uh, thank you for joining in.

00:08:04.335 --> 00:08:08.385
So if you've enjoyed this, uh, uh,
very much appreciated, we'll try

00:08:08.385 --> 00:08:10.605
to continue with, uh, enjoyable.

00:08:11.025 --> 00:08:11.955
Uh, content.

00:08:12.855 --> 00:08:16.575
It, it has increased, uh, over
the time, not unexpectedly.

00:08:16.575 --> 00:08:18.915
We had zero, uh, a year ago.

00:08:19.155 --> 00:08:25.275
And, um, uh, so over time, somewhat
linearly increasing the number of them for

00:08:25.275 --> 00:08:29.775
each, the number that consume it within
the first couple days, the, in the first

00:08:29.775 --> 00:08:32.775
couple weeks, we do have global listeners.

00:08:32.775 --> 00:08:36.375
About 37% of the watches
were the United States.

00:08:36.525 --> 00:08:38.385
6% were Australia.

00:08:38.930 --> 00:08:42.590
Uh, many colleagues and, uh,
uh, friends in Australia.

00:08:42.590 --> 00:08:43.760
So appreciate that.

00:08:44.030 --> 00:08:49.130
The UK is about 6%, India's
about 6% Germany, about 4%

00:08:49.130 --> 00:08:51.530
Switzerland, uh, and down.

00:08:51.530 --> 00:08:53.270
So, uh, Canada.

00:08:53.270 --> 00:08:58.460
So really global representation, which
is, which is also very, very humbling.

00:08:58.970 --> 00:09:04.220
And I guess one really interesting
thing, in December we were listed

00:09:04.280 --> 00:09:07.220
on the top 10 popular shows.

00:09:08.345 --> 00:09:12.095
For the mathematics
category of Apple Podcasts.

00:09:12.095 --> 00:09:17.975
So we ended up number seven on the
math category for Apple Podcasts.

00:09:17.975 --> 00:09:20.225
And so that was, that
was really neat to see.

00:09:20.285 --> 00:09:24.485
So again, thank you on that and maybe
I'll come back and, and update you

00:09:24.485 --> 00:09:27.485
on the various, uh, uh, metrics.

00:09:27.485 --> 00:09:28.475
How are we doing here?

00:09:28.925 --> 00:09:31.235
The, I, I've got some neat ones coming up.

00:09:31.235 --> 00:09:34.035
I think some really neat
guests, uh, within that.

00:09:35.025 --> 00:09:37.305
So today, what, what did I
want to talk about today?

00:09:37.305 --> 00:09:40.245
It's really sort of front and
center of my mind that I'm

00:09:40.245 --> 00:09:41.475
getting on an airplane today.

00:09:41.475 --> 00:09:45.825
By the way, this is Austin, Texas
in the background, uh, here.

00:09:45.825 --> 00:09:50.145
And I get on a plane in about
three hours and I'm headed to dc.

00:09:50.145 --> 00:09:52.350
I have a meeting with the FDA tomorrow.

00:09:52.740 --> 00:09:54.600
Uh, with a sponsor.

00:09:54.600 --> 00:09:56.220
This is all a sponsor.

00:09:56.670 --> 00:09:59.790
And I thought I'd give you
a bit of a, what is this?

00:10:00.180 --> 00:10:02.130
What, what happens at these meetings?

00:10:02.130 --> 00:10:03.060
What are they about?

00:10:03.360 --> 00:10:07.200
Little bit of advice, uh, from it, and so.

00:10:08.475 --> 00:10:12.855
This is a type C meeting, and
I'm not a regulatory expert.

00:10:13.065 --> 00:10:15.585
I'm not gonna give you all
the, the, the differences of

00:10:15.585 --> 00:10:17.625
type A, type B, type C meeting.

00:10:17.895 --> 00:10:20.985
I think this is the most
general of the meetings where

00:10:20.985 --> 00:10:22.815
you can go for general advice.

00:10:23.115 --> 00:10:30.915
This, this one particular is proposing
a phase three trial for a, uh, for a

00:10:30.915 --> 00:10:32.710
biologic actually, but, uh, for a drug.

00:10:33.900 --> 00:10:38.160
And that's a common type meeting
that I, I experience is trying to

00:10:38.160 --> 00:10:40.230
get approval for a development plan.

00:10:40.890 --> 00:10:43.830
There are other meetings where you're
trying to get rid of a clinical hold.

00:10:43.830 --> 00:10:47.520
You have specific issues, uh,
and those give different names.

00:10:47.520 --> 00:10:51.750
But so focus on this 'cause it's, it's
typically the type of meeting I go to.

00:10:52.350 --> 00:10:56.245
So I've, I've been doing this at Berry
Co Consultants slightly over 25 years.

00:10:57.195 --> 00:10:58.650
Been to many of these meetings.

00:10:58.685 --> 00:11:00.030
I, I don't count them.

00:11:00.030 --> 00:11:01.200
I should count them.

00:11:02.400 --> 00:11:05.760
It's more than a hundred,
uh, of these meetings.

00:11:06.120 --> 00:11:10.110
They've been at the Center for
Biologics, center for Devices,

00:11:10.260 --> 00:11:13.920
center for um, drugs within this.

00:11:13.920 --> 00:11:16.200
So multiple types of meetings.

00:11:16.200 --> 00:11:21.480
And so just a viewpoint of, of mine,
uh, on what these meetings look like.

00:11:22.695 --> 00:11:26.925
Okay, and I'm gonna do this without
telling you anything about who the

00:11:26.925 --> 00:11:28.635
sponsor is for the meeting tomorrow.

00:11:28.635 --> 00:11:29.925
Any details of that?

00:11:30.195 --> 00:11:33.315
Uh, as I, I, I can't
give you those details.

00:11:33.405 --> 00:11:37.665
So just high level views, generic
aspects of these meetings.

00:11:38.325 --> 00:11:42.375
So many of these meetings that
I'm involved in is we're, we're

00:11:42.375 --> 00:11:45.165
proposing, uh, a particular design.

00:11:45.795 --> 00:11:48.795
Now, these meetings can be hybrid.

00:11:48.810 --> 00:11:48.870
It.

00:11:49.350 --> 00:11:54.000
They can be written feedback only
from FDA or they can be in person.

00:11:54.390 --> 00:11:56.340
This is an in-person meeting.

00:11:56.520 --> 00:11:59.460
Interestingly, by the way, the
government might be shut down.

00:11:59.460 --> 00:12:00.840
These meetings happen.

00:12:01.335 --> 00:12:07.305
And, uh, there's a risk, uh, that the
government is shut down because sponsors

00:12:07.305 --> 00:12:11.535
fund these, they come from the user
fees, and so they're paying for these.

00:12:11.535 --> 00:12:13.245
And so these meetings happen.

00:12:13.245 --> 00:12:16.755
They don't get canceled when
the government is shut down.

00:12:17.025 --> 00:12:21.915
Other activities, uh, FDA people can't
travel to meetings and other activities,

00:12:21.915 --> 00:12:24.225
but user fee funded things they have.

00:12:24.660 --> 00:12:26.220
So there, there's no risk of that.

00:12:26.970 --> 00:12:31.920
The typical scenario is you submit
a meeting request to the FDA.

00:12:32.760 --> 00:12:34.890
We want to have a meeting with the FDA.

00:12:34.890 --> 00:12:40.410
You can request in person, you can request
written feedback only, and then the FDA

00:12:40.410 --> 00:12:42.510
decides what kind of meeting you get.

00:12:42.780 --> 00:12:45.750
They can say, we're only going
to provide written feedback.

00:12:46.665 --> 00:12:49.215
They can say, sure, we'll
do an in-person meeting.

00:12:49.215 --> 00:12:53.085
They can do a hybrid meeting
where it's, it's all online.

00:12:53.085 --> 00:12:54.675
Used to be telephone calls.

00:12:54.975 --> 00:12:59.805
So I'm old enough that many of these were
done by telephone call only within that.

00:12:59.805 --> 00:13:05.415
But now they are done through a a, a zoom
type teams type meeting as a possibility.

00:13:05.415 --> 00:13:08.235
So they can say, no, we don't
want to do an in person meeting.

00:13:08.235 --> 00:13:09.370
We're only giving you written feedback.

00:13:10.500 --> 00:13:16.110
So you submit a request for a particular
day, timing of this within 60 days.

00:13:16.110 --> 00:13:19.050
So a coup, you need to give them
a couple months notice, but you

00:13:19.050 --> 00:13:21.600
also submit questions to the FDA.

00:13:22.440 --> 00:13:24.750
So these meetings are not.

00:13:25.395 --> 00:13:26.745
FDA grilling you.

00:13:26.745 --> 00:13:31.275
You are asking questions of the
FDA, and I'll talk more about that.

00:13:31.275 --> 00:13:35.985
But you list out the questions and
these are typically, there may be rules

00:13:35.985 --> 00:13:39.915
to this, but these are typically four
to eight questions you ask of the FDA.

00:13:39.915 --> 00:13:43.160
This is not 50 questions
to them, but four.

00:13:44.075 --> 00:13:47.315
And yes, some questions
are, here's a question.

00:13:47.315 --> 00:13:51.185
Here's part A, B, C, D, but
largely four to eight questions

00:13:51.185 --> 00:13:52.505
that you ask of the FD, A.

00:13:52.505 --> 00:13:54.035
And that goes with the meeting request.

00:13:54.035 --> 00:13:58.445
And so they know who should put at the
meeting, the, the topic of it, decide what

00:13:58.445 --> 00:14:00.665
kind of meeting it should be within that.

00:14:01.905 --> 00:14:05.805
Um, and then within 30 days
you submit more detail to them.

00:14:06.075 --> 00:14:10.575
So you submit what's called a
briefing book that has the sponsor

00:14:10.575 --> 00:14:14.025
position on, on the questions.

00:14:14.175 --> 00:14:20.565
So is this design appropriate for
a phase three trial for, uh, uh,

00:14:20.565 --> 00:14:22.635
getting approval of this drug?

00:14:23.775 --> 00:14:26.385
And that might be one
of the questions on it.

00:14:26.685 --> 00:14:29.145
Uh, it might be an adaptive design.

00:14:29.745 --> 00:14:33.705
Does the FDA agree that the adaptive
design controls type one error or

00:14:33.705 --> 00:14:38.325
that the Bayesian borrowing or, or
whatever it is, uh, there may be a

00:14:38.325 --> 00:14:40.305
question specifically about that.

00:14:40.305 --> 00:14:47.655
Does FDA agree that this is an appropriate
way, uh, to do it and you put a position

00:14:47.655 --> 00:14:49.875
you support why you're doing it?

00:14:50.775 --> 00:14:56.445
You don't give, I generally,
everything I say here is generally

00:14:56.505 --> 00:14:59.835
somebody's gonna say, Hey, I had
a meeting where X, Y, Z happened.

00:15:00.375 --> 00:15:01.665
I'm sure, I'm sure.

00:15:01.665 --> 00:15:06.375
So these are very general comments,
but you, you don't throw, you don't

00:15:06.375 --> 00:15:10.065
submit five designs and say, FDA,
what, which one do you like better?

00:15:10.065 --> 00:15:14.625
You submit your proposed design
and you support it, and you, you

00:15:14.625 --> 00:15:16.695
present why you support that design.

00:15:17.655 --> 00:15:20.625
Within it, uh, that's an
important part of this.

00:15:20.625 --> 00:15:23.655
You're asking them those questions
around it and you're supporting it.

00:15:25.385 --> 00:15:28.920
The, the, so writing the briefing
book is a big part of it.

00:15:28.980 --> 00:15:32.550
Uh, submitting a a synopsis
is part of this submitting.

00:15:32.670 --> 00:15:36.390
You don't submit a statistical
analysis plan this early, but you

00:15:36.390 --> 00:15:40.770
give the details needed to support
the various aspects of the design.

00:15:41.010 --> 00:15:43.335
And I'm gonna focus on the
statistical parts of this.

00:15:43.425 --> 00:15:47.070
And, and you shouldn't be
surprised by, by that part of it.

00:15:47.760 --> 00:15:52.920
Now, high level is what happens
is, so we have a meeting.

00:15:53.940 --> 00:16:00.180
Tomorrow, uh, it's a Wednesday,
and we got feedback from the FDA.

00:16:00.180 --> 00:16:05.610
The first contact about their responses
to them don't happen at the meeting.

00:16:06.840 --> 00:16:07.290
It.

00:16:07.290 --> 00:16:08.850
They provide written feedback.

00:16:08.850 --> 00:16:12.810
We got that Friday afternoon,
and I don't know the rules.

00:16:12.810 --> 00:16:18.600
It's like within 72 hours or 48 hours
or something that they're required to

00:16:18.600 --> 00:16:21.660
respond in writing about your questions.

00:16:21.990 --> 00:16:27.360
And so we got those late Friday
afternoon and um, uh, within that,

00:16:27.360 --> 00:16:30.270
and you get that feedback now.

00:16:31.005 --> 00:16:34.755
I'll spend a little bit of time on what
kind of happens, uh, in the sponsor.

00:16:34.995 --> 00:16:38.385
And it typically happens
once that feedback gets in.

00:16:39.435 --> 00:16:43.695
The one of the reasons I, I think,
for example, they have, they may

00:16:43.695 --> 00:16:47.695
have written up all those responses
and had them two weeks ago.

00:16:49.200 --> 00:16:53.670
But they don't submit them, and I
think the reason why they wait till

00:16:53.670 --> 00:16:59.730
kind of to the end is they don't want
the sponsor creating new material

00:16:59.730 --> 00:17:01.155
for them to have to evaluate.

00:17:01.440 --> 00:17:01.500
It.

00:17:02.730 --> 00:17:07.020
And so the meeting, you can't
bring new material to the meeting.

00:17:07.680 --> 00:17:12.300
Everything you submitted, the briefing
book is fair game for questions for that.

00:17:12.300 --> 00:17:16.620
You can ask them to clarify things
about their position on it, but

00:17:16.620 --> 00:17:20.250
you can't say, Hey, we went back
and we found this new paper.

00:17:21.060 --> 00:17:24.090
Um, we, we, we created this new design.

00:17:24.090 --> 00:17:25.320
What do you think of this?

00:17:25.590 --> 00:17:26.970
They haven't had time to review it.

00:17:26.970 --> 00:17:29.130
They, they won't answer those
questions in the meeting.

00:17:29.160 --> 00:17:30.180
No new material.

00:17:30.180 --> 00:17:32.760
And I think that's why the
written feedback comes so late.

00:17:33.510 --> 00:17:33.870
Sure.

00:17:33.870 --> 00:17:37.560
Sponsors would've it like to have it
two weeks earlier, uh, to, to really

00:17:37.560 --> 00:17:39.930
plan and, and scenario around that.

00:17:39.960 --> 00:17:41.310
So that comes relatively late.

00:17:41.310 --> 00:17:42.690
And I think there's a purpose to that.

00:17:43.680 --> 00:17:47.520
These meeting, this meeting's one
hour, and I think most of these, every

00:17:47.520 --> 00:17:49.220
once in a while, there's a, there's a.

00:17:49.740 --> 00:17:52.320
Hour and a half meeting, but I
think the typical meeting is one

00:17:52.320 --> 00:17:55.290
hour and it is strictly one hour.

00:17:55.740 --> 00:17:59.130
So I've been in a number of these
meetings where I think the sponsor

00:17:59.130 --> 00:18:02.130
largely wastes time and I'll
come back to that a little bit.

00:18:02.160 --> 00:18:04.560
And when that hour's up, it's over.

00:18:04.650 --> 00:18:07.710
The project manager takes
you and you leave the meeting

00:18:07.710 --> 00:18:09.240
or the zoom meeting ends.

00:18:09.390 --> 00:18:10.110
It's over.

00:18:10.800 --> 00:18:11.310
You did.

00:18:11.310 --> 00:18:14.340
You wanted to spend 20 more
minutes telling them how great

00:18:14.340 --> 00:18:16.440
your drug was and it's over.

00:18:16.470 --> 00:18:17.310
That's your shot.

00:18:18.255 --> 00:18:21.105
So it's one hour is really, really short.

00:18:21.285 --> 00:18:24.885
And so planning that what you're
gonna do in that appropriately.

00:18:25.545 --> 00:18:29.145
The other part of this is, this
is your meeting as the sponsor.

00:18:30.960 --> 00:18:33.180
This is not the FDA's meeting.

00:18:33.270 --> 00:18:38.010
They don't bring the agenda where they
want to talk more about X, Y, or Z.

00:18:38.250 --> 00:18:42.180
They show up and you submit an
agenda and you get to ask the

00:18:42.180 --> 00:18:46.560
questions as the sponsor, uh,
within that, and it's your meeting.

00:18:46.560 --> 00:18:50.280
If you think their responses
that they supplied to the first

00:18:50.280 --> 00:18:51.870
three questions are sufficient.

00:18:52.305 --> 00:18:53.955
You don't even talk about
those in the meeting.

00:18:54.435 --> 00:18:55.695
You don't need to go through them.

00:18:55.695 --> 00:18:58.915
And, but, but question four,
that's the, that, that's really,

00:18:58.915 --> 00:19:02.175
what you want clarification, you
want further discussion on that?

00:19:02.175 --> 00:19:05.145
That's open game and you
might submit to them.

00:19:05.355 --> 00:19:08.805
So then largely you put an
agenda together, you, you

00:19:08.805 --> 00:19:10.315
you submit back to the FDA.

00:19:10.335 --> 00:19:13.305
We would like to talk about
question four and question seven.

00:19:13.335 --> 00:19:14.325
Those are the topics.

00:19:14.475 --> 00:19:18.615
So the FDA knows sort of what's gonna
happen at the meeting within that.

00:19:18.615 --> 00:19:19.105
And again, they.

00:19:19.585 --> 00:19:22.345
They don't want new material because
they don't have time to review

00:19:22.345 --> 00:19:24.295
it with within that timeframe.

00:19:25.225 --> 00:19:30.835
Um, you can also cancel the meeting
at that point, the written response.

00:19:30.835 --> 00:19:34.225
If you got a yes to every question,
your design looks great, you.

00:19:36.105 --> 00:19:36.945
Don't need the meeting.

00:19:36.975 --> 00:19:40.335
And we've had those, it, it's not
that uncommon, actually, I don't know,

00:19:40.335 --> 00:19:45.345
25% of meetings might be canceled,
that the response is sufficient.

00:19:45.345 --> 00:19:47.805
There's no clarifications
needed within that.

00:19:47.805 --> 00:19:49.485
And, and that's, that's there.

00:19:49.905 --> 00:19:53.805
I I think generally sponsors don't like
to cancel the meeting anytime they get in

00:19:53.805 --> 00:19:56.595
front of FDA, that they want to do that.

00:19:56.865 --> 00:19:58.545
And, uh, but.

00:19:59.355 --> 00:20:02.715
Um, that there are times when
absolutely we've got our responses.

00:20:02.715 --> 00:20:03.585
We know what to do.

00:20:03.585 --> 00:20:05.925
We don't, we don't have
any additional questions.

00:20:06.105 --> 00:20:10.455
Sometimes my part of the meeting
is canceled, so the design, the

00:20:10.455 --> 00:20:14.715
statistical part, they want to talk
about manufacturing or they want to

00:20:14.715 --> 00:20:20.265
talk about a particular toxicology
test, and that that's the only thing,

00:20:20.265 --> 00:20:22.245
the question on that, that they have.

00:20:22.395 --> 00:20:25.965
And so they, they, they change
who's going and, and I, and

00:20:25.965 --> 00:20:27.645
I might not go within that.

00:20:28.035 --> 00:20:30.585
I am going to the meeting
tomorrow, for example.

00:20:31.725 --> 00:20:32.175
Okay.

00:20:32.535 --> 00:20:37.095
So that's typically, uh, uh, kind
of what these meetings look like.

00:20:37.095 --> 00:20:42.255
So sort of high level, um, uh,
the, what does the prep look like?

00:20:42.255 --> 00:20:47.955
So once you get that response, it's, it's
reading the responses, it's consuming it.

00:20:48.495 --> 00:20:54.195
There's, these are worded
incredibly precisely, and

00:20:54.195 --> 00:20:56.445
the reason why I say that is.

00:20:56.460 --> 00:21:02.280
There's, um, there are different
kinds of no in it and there are

00:21:02.280 --> 00:21:04.050
different kinds of advice in it.

00:21:04.470 --> 00:21:09.780
The FDA can say, absolutely no, we will
not allow you to do that to patients.

00:21:09.840 --> 00:21:11.280
And it's a hard no.

00:21:12.990 --> 00:21:19.920
You's also softer nos, and there's,
you can do that and it's your risk.

00:21:20.400 --> 00:21:24.450
We suggest you don't do
that and you do X, Y, and z.

00:21:25.980 --> 00:21:31.830
There's also, um, largely this sort
of feedback that we think you should

00:21:31.830 --> 00:21:34.155
do this sort of other thing, but.

00:21:35.055 --> 00:21:36.885
They're not saying no kind of thing.

00:21:37.155 --> 00:21:40.785
So it's consuming all of this
and largely what do we get about

00:21:40.785 --> 00:21:46.065
this feedback and deciding do we
need, do we need clarification?

00:21:46.065 --> 00:21:46.905
What do we need?

00:21:47.115 --> 00:21:48.375
What are the parts of this?

00:21:48.375 --> 00:21:52.575
And this is the real important part
and it's, it's kind of hectic because

00:21:52.575 --> 00:21:54.495
you need to get back to the FDA.

00:21:54.525 --> 00:21:55.785
They needed to submit.

00:21:56.130 --> 00:21:59.790
Yesterday morning, I think, uh,
what they want to do and any

00:21:59.790 --> 00:22:03.270
new, any slides they wanna show
to the FDA had to bid yesterday.

00:22:03.300 --> 00:22:05.400
So they need to really decide,
do we have the meeting?

00:22:05.910 --> 00:22:07.410
What do we want to talk about?

00:22:07.530 --> 00:22:13.650
And a lot of that is, um,
what, what's a win here?

00:22:13.650 --> 00:22:15.900
What are we trying to accomplish by that?

00:22:16.380 --> 00:22:20.220
And it, by having the
meeting, so there's this huge.

00:22:20.700 --> 00:22:21.600
Frequent calls.

00:22:21.600 --> 00:22:24.180
Lots of discussion once
that thing comes in.

00:22:24.180 --> 00:22:27.270
And of course it was over the
weekend this time, uh, having these

00:22:27.270 --> 00:22:30.510
various comments to it and really
deciding what are we gonna do?

00:22:30.510 --> 00:22:32.910
What are the things that
we want to talk about?

00:22:32.910 --> 00:22:36.600
Get clarification, and then really
identifying what's a win here?

00:22:36.630 --> 00:22:38.640
Why, why do we need clarification?

00:22:38.640 --> 00:22:40.350
What do we want to have happen?

00:22:40.800 --> 00:22:41.820
What's a good outcome?

00:22:41.820 --> 00:22:43.110
What's a bad outcome?

00:22:43.380 --> 00:22:47.250
These parts all go into to,
to then decide, and you, okay,

00:22:47.340 --> 00:22:48.780
here's where we're going with it.

00:22:49.665 --> 00:22:54.345
Um, within that, so that this kind
of all happens hurriedly and that's

00:22:54.405 --> 00:22:56.145
for me about to get on a plane.

00:22:56.145 --> 00:22:56.895
That's all happened.

00:22:56.895 --> 00:23:01.845
We've had these discussions
within that it, it typically, um.

00:23:02.715 --> 00:23:05.355
Typically then you, you
meet with the sponsor.

00:23:05.355 --> 00:23:08.385
And by the way, I've done
this with academic sponsors.

00:23:08.655 --> 00:23:11.805
I've done this with patient
organizations being the sponsor

00:23:11.805 --> 00:23:15.795
who submitted with pharmaceutical
companies, a medical device company.

00:23:15.795 --> 00:23:21.075
So sponsors come in many, many
flavors within this, typically

00:23:21.075 --> 00:23:22.695
we, you have pre-meetings to it.

00:23:22.695 --> 00:23:25.275
There's some weird aspects to
it that I'm not actually there

00:23:25.275 --> 00:23:27.495
today within it, but we are.

00:23:27.970 --> 00:23:31.090
Tomorrow morning, we
have multiple meetings.

00:23:31.090 --> 00:23:34.330
We're gonna have, you know, four
or five hours where we're gonna go

00:23:34.330 --> 00:23:35.740
through what that all looks like.

00:23:35.740 --> 00:23:37.810
So we'll, we'll, we'll
meet tomorrow morning.

00:23:38.380 --> 00:23:42.880
Within that, we, we largely, for this
one, there's only a limited set of things

00:23:42.880 --> 00:23:44.600
we needed to discuss and so we, we.

00:23:44.890 --> 00:23:48.340
Didn't necessarily need to show up a
full day in advance, but a lot of times,

00:23:48.340 --> 00:23:51.970
this might even be two days in advance,
you're planning out this one hour meeting

00:23:52.210 --> 00:23:55.180
and what do we want to talk about and
how we're gonna do it, and all of that.

00:23:56.320 --> 00:24:01.210
Um, and then typically, you all,
you, you're all at a meeting place.

00:24:01.510 --> 00:24:06.760
You get on a big bus and you drive to
the white Oak facility in Washington, DC.

00:24:07.785 --> 00:24:08.445
Yes.

00:24:08.655 --> 00:24:09.705
Suit and tie.

00:24:10.005 --> 00:24:13.005
Uh, within that I did have a sponsor once.

00:24:13.005 --> 00:24:19.215
Tell me, and I'll chuckle at this,
uh, not to wear your expensive Italian

00:24:19.215 --> 00:24:25.215
suit, and I chuckle because I'm a
statistician and I own one suit.

00:24:26.070 --> 00:24:30.690
Um, and it's not an expensive Italian
suit, uh, within it, by the way.

00:24:30.690 --> 00:24:34.290
I have multiple kids getting married and
all that, so I did buy a tuxedo, and I

00:24:34.290 --> 00:24:38.280
don't wear a tuxedo to the FDA, but I do
own a tuxedo for the various weddings.

00:24:38.295 --> 00:24:44.595
Uh, uh, I have lots of cousins and I,
nephews and nieces and lots of weddings,

00:24:44.595 --> 00:24:49.575
and so it's, it's a nice thing to have,
but I, I own one suit, so, uh, I was at

00:24:49.575 --> 00:24:54.255
least given that, and I think they're
worried about, um, and I have no idea,

00:24:54.255 --> 00:25:00.285
but, but that, uh, uh, doctors show
up, high paid doctors show up to a

00:25:00.285 --> 00:25:04.005
place, and, and FDA makes less money.

00:25:04.530 --> 00:25:09.855
Uh, within this, they then, if you got a
position at a sponsor, for example, um,

00:25:12.390 --> 00:25:14.040
you go to the White Oak facility.

00:25:14.700 --> 00:25:19.200
Building 22 is where the Cedar
and Seber meetings happen.

00:25:19.350 --> 00:25:23.430
All the, the, the, the
conference rooms there for it.

00:25:23.670 --> 00:25:28.170
And building 66 is where
you go to the CDRH meetings.

00:25:28.170 --> 00:25:30.780
That's sort of down the hill
a little bit from there.

00:25:31.350 --> 00:25:33.300
Uh, you drive into this facility.

00:25:33.360 --> 00:25:36.480
Uh, it's interesting, I
predate this facility where.

00:25:37.110 --> 00:25:41.730
The, the meetings took place at
sort of sporadic office buildings

00:25:41.940 --> 00:25:43.560
before the White Oak facility.

00:25:43.560 --> 00:25:45.270
The White Oak facility's awesome.

00:25:45.720 --> 00:25:47.970
It'd be great if it was
closer to the Metro.

00:25:48.210 --> 00:25:53.100
Um, I, I think they actually have buses
for employees there, but, uh, within

00:25:53.250 --> 00:25:54.905
it, it's now very highly secured.

00:25:56.025 --> 00:25:59.955
And so it's, it's, there's
a wide fence to this.

00:25:59.955 --> 00:26:04.785
You go through, uh, uh, security
comes out, they're looking under your

00:26:04.785 --> 00:26:06.255
car, they're checking everything.

00:26:06.255 --> 00:26:08.025
So it's a very, very secure building.

00:26:08.025 --> 00:26:11.715
Once you get in and then you drive
to the building, and then further

00:26:11.715 --> 00:26:16.845
you go in there and you go through
additional security at that point, um.

00:26:16.900 --> 00:26:22.780
And we, I interestingly, they, they,
they can be almost, I, it's changed

00:26:22.780 --> 00:26:26.590
a little bit, but four or five years
ago, they were almost, I'll say, more

00:26:26.590 --> 00:26:32.320
thorough than the airport, and they
asked people to pull every single

00:26:32.320 --> 00:26:34.570
cord out of their bags and all this.

00:26:34.570 --> 00:26:39.550
So you'd have these clinicians would
show up with this big carry bag, and

00:26:39.550 --> 00:26:41.560
they're pulling everything out of it.

00:26:41.750 --> 00:26:44.390
Extremely slow within this.

00:26:44.390 --> 00:26:45.650
Everybody's checked in.

00:26:45.650 --> 00:26:46.760
IDs are checked.

00:26:46.910 --> 00:26:49.700
Foreign visitors,
passports are gone through.

00:26:49.940 --> 00:26:52.190
So, uh, hit number one.

00:26:52.220 --> 00:26:53.600
Don't bring anything to the meeting.

00:26:53.600 --> 00:26:54.560
I don't bring anything.

00:26:54.560 --> 00:26:56.300
I leave everything on the bus.

00:26:56.390 --> 00:26:59.810
Uh, I bring a, a pad
of paper and a pencil.

00:27:00.335 --> 00:27:05.340
A a, a pen or a pencil and I don't bring
my computer, I don't bring any coin cords.

00:27:05.520 --> 00:27:08.430
I generally have my phone to
keep track of time within that.

00:27:08.610 --> 00:27:10.920
Really easy to go through security with.

00:27:10.950 --> 00:27:15.780
Within that, um, by the way, when
you go into these facilities,

00:27:16.140 --> 00:27:19.650
they have big pictures of the
president of the United States.

00:27:20.205 --> 00:27:25.035
The head of Health and Human Services
and the commissioner of the FDA, maybe

00:27:25.035 --> 00:27:30.405
the Deputy commissioner of the FDA,
but these big prominent pictures,

00:27:30.405 --> 00:27:34.605
when you go in, uh, of that, um,

00:27:36.855 --> 00:27:41.380
then right on time, project
manager takes you to your meeting.

00:27:42.540 --> 00:27:44.820
It starts right on time.

00:27:45.540 --> 00:27:50.070
And again, that meeting's yours,
you can't record the meeting.

00:27:50.490 --> 00:27:54.030
Uh, within that, uh, it is very explicit.

00:27:54.030 --> 00:27:55.290
There's no recording of it.

00:27:55.290 --> 00:27:59.190
Minutes are taken within that, and
the minutes are always reviewed

00:27:59.190 --> 00:28:02.370
and they, they, they create
official minutes of the meeting.

00:28:02.370 --> 00:28:04.590
But no, no recordings of it are done.

00:28:04.920 --> 00:28:06.990
And the FDA team is typically there.

00:28:07.350 --> 00:28:12.990
You go in right out there at 1 0 1
and uh, one o'clock, the FDA team

00:28:12.990 --> 00:28:14.370
is there and it's your meeting.

00:28:15.090 --> 00:28:17.400
And there might be introductions.

00:28:17.400 --> 00:28:18.510
Typically in person.

00:28:18.510 --> 00:28:22.740
There's introductions of the people,
uh, hybrid meetings typically you, you

00:28:22.740 --> 00:28:24.420
introduce yourself and you start talking.

00:28:24.420 --> 00:28:27.090
'cause there might be 40
people on this part of it.

00:28:28.080 --> 00:28:30.660
And the discussion happens
and it's your meeting.

00:28:30.660 --> 00:28:33.570
You can ask questions, you
can present within that.

00:28:33.570 --> 00:28:36.090
And I'll talk more, uh, about that.

00:28:36.540 --> 00:28:41.130
The other part is the meeting ends
at one hour, and I've been in those

00:28:41.130 --> 00:28:45.870
situations where essentially the FDA
walks out and the project manager takes

00:28:45.870 --> 00:28:50.910
you out and you, you again, you, you
want to say more and you can't now.

00:28:51.480 --> 00:28:56.580
I, I do, I I have seen cases where hall
discussion happens and the FDA wants to

00:28:56.580 --> 00:29:01.200
make sure they tell 'em X, Y, or Z, but
you can't count on that and it's probably

00:29:01.200 --> 00:29:02.940
shouldn't be happening, sort of thing.

00:29:03.150 --> 00:29:07.380
But they also have the room scheduled,
so they are going to get you out.

00:29:07.380 --> 00:29:10.830
And so using that one hour
appropriately, it's huge.

00:29:11.280 --> 00:29:13.560
Uh, within it, you're escorted out.

00:29:13.580 --> 00:29:16.970
And then typically after the meeting,
you might go back to the hotel, discuss

00:29:16.970 --> 00:29:18.890
what they said, where should we go?

00:29:18.890 --> 00:29:21.080
What should we do, what
work needs to be done?

00:29:21.680 --> 00:29:25.460
Typically, there's a rush to the
airport, uh, within it, a bunch of

00:29:25.460 --> 00:29:27.080
people headed off to the airports.

00:29:27.080 --> 00:29:30.800
You try to get out, people get back,
uh, within that, and that's kind of

00:29:30.800 --> 00:29:35.360
the day, and that's kind of what my, my
day's going to look like within that.

00:29:38.505 --> 00:29:40.185
I, I wanna be clear.

00:29:40.215 --> 00:29:46.215
So, so suggestions for these
meetings within that, and I've

00:29:46.215 --> 00:29:48.015
seen these meetings go very well.

00:29:48.015 --> 00:29:50.325
I've seen these, me
meetings go quite poorly.

00:29:50.655 --> 00:29:51.915
Um, within it.

00:29:51.915 --> 00:29:54.525
I've probably done a
bad job in some of them.

00:29:54.915 --> 00:29:55.995
Uh, within that.

00:29:56.865 --> 00:29:59.055
The FDA is incredibly smart.

00:29:59.235 --> 00:30:01.515
The, by the way, the FDA is really good.

00:30:02.160 --> 00:30:06.300
And these are people that are
doing this for the right reason.

00:30:06.300 --> 00:30:07.350
They're dedicated.

00:30:07.530 --> 00:30:11.820
They want to see safe and effective
treatments go to patients.

00:30:12.360 --> 00:30:20.280
There's nobody there that is just a hard
ass and my job's to make it hard on you.

00:30:20.490 --> 00:30:21.870
It's not that at all.

00:30:22.200 --> 00:30:27.210
They want you to be
successful a hundred percent.

00:30:27.210 --> 00:30:28.290
It's good for patients.

00:30:28.290 --> 00:30:29.375
They want you to be successful.

00:30:30.315 --> 00:30:30.975
Within that.

00:30:31.395 --> 00:30:33.525
And so that's a big part of it.

00:30:33.585 --> 00:30:36.345
They're on your side now.

00:30:36.555 --> 00:30:41.205
You can do things that make it, they're
not on your side anymore, and you

00:30:41.205 --> 00:30:43.605
build up walls and you create things.

00:30:43.605 --> 00:30:48.525
For example, starting the meeting where
you're about to run a phase three trial,

00:30:48.525 --> 00:30:53.955
reminding them how it's so utterly
clear how good your drug is in it.

00:30:53.955 --> 00:30:57.765
You, you don't know why the
FDA hasn't approved it yet now

00:30:57.765 --> 00:30:59.205
means they're not on your side.

00:31:00.525 --> 00:31:04.695
If, and, and there are circumstances where
maybe you've run one phase three trial,

00:31:04.695 --> 00:31:08.685
you think the drug should be approved,
whatever, and you, you disagree with

00:31:08.685 --> 00:31:14.205
that FDA meeting that's creating a fence
between them and there are lots of ways in

00:31:14.205 --> 00:31:16.665
which these fences are created within it.

00:31:16.785 --> 00:31:20.895
Now they're not on your side because
you're forcing them to tell you again, no.

00:31:22.200 --> 00:31:25.920
And, and now there's this
sort of wall aspect of it.

00:31:25.920 --> 00:31:31.290
But, but again, they all want safe and
effective treatments to be out there.

00:31:31.620 --> 00:31:36.540
We're all patients, we're all humans,
and they're incredibly good at this.

00:31:37.320 --> 00:31:41.880
Now the other thing that I see that
I struggle with, you have a one hour

00:31:41.880 --> 00:31:48.675
meeting and way too much time is
spent and people can disagree with me.

00:31:49.995 --> 00:31:56.235
Telling the FDA, how bad the disease
is, how much patients need a treatment.

00:31:57.825 --> 00:31:59.145
They know this.

00:31:59.955 --> 00:32:04.845
They absolutely know this, and
the fact that patients need

00:32:04.845 --> 00:32:08.505
something doesn't mean that we're
now gonna approve your treatment.

00:32:08.505 --> 00:32:09.825
They don't need bad treatments.

00:32:09.885 --> 00:32:12.645
They don't need ineffective
treatments, non-safe treatments.

00:32:13.920 --> 00:32:15.630
It's largely a waste of time.

00:32:15.690 --> 00:32:19.740
And I'll sit there and this will go
on for 10 or 15 minutes where this

00:32:19.740 --> 00:32:21.360
is done and there's no point to it.

00:32:21.780 --> 00:32:27.900
Um, and it's actually, I think it's a
little insulting that they think that

00:32:27.900 --> 00:32:32.190
you need to go through those details
and remind them of, of how bad this

00:32:32.190 --> 00:32:37.260
disease is, for example, and how many
patients need this and, and that now.

00:32:37.620 --> 00:32:41.490
Obviously to back up there,
there are strange diseases.

00:32:41.640 --> 00:32:46.620
There are strange scenarios where it's
appropriate to review the information

00:32:46.620 --> 00:32:51.090
out there and the FDA might not have an
expert in this situation, by the way.

00:32:51.090 --> 00:32:54.810
They do bring in experts, they bring
in external people that are working

00:32:54.810 --> 00:32:59.190
for the FDA that know this particular
disease, this subset, that sort of thing.

00:32:59.370 --> 00:33:03.810
So, you know, generally there,
there is that, but if you're

00:33:03.810 --> 00:33:05.910
going in there in a relatively.

00:33:06.600 --> 00:33:11.490
Known disease, even a rare disease,
you're going in there and a LS and saying,

00:33:11.640 --> 00:33:15.930
my meeting's not for a LS, by the way,
and saying patients need something they

00:33:15.930 --> 00:33:18.150
know a LS, and I think it's insulting.

00:33:18.150 --> 00:33:21.600
So, and again, what's the point to it?

00:33:22.560 --> 00:33:24.600
You're not moving the ball forward.

00:33:24.840 --> 00:33:27.480
You have one hour to try to figure out.

00:33:27.480 --> 00:33:31.110
So spend a lot of time
figuring out what success is.

00:33:32.040 --> 00:33:33.895
You know, what do you
want out of this meeting?

00:33:34.950 --> 00:33:41.130
Um, uh, within that and then be very,
sort of driven by that particular goal.

00:33:41.130 --> 00:33:45.750
And I'll present examples where I, I
probably didn't do very well in that.

00:33:46.050 --> 00:33:47.430
Uh, within it.

00:33:48.360 --> 00:33:54.810
Another way to prevent this sort of,
they're not on your side anymore, is

00:33:54.810 --> 00:33:56.940
when you ask them impossible questions.

00:33:57.225 --> 00:34:02.925
It drives, drives me nuts when
you ask them impossible questions.

00:34:03.405 --> 00:34:08.715
If we see success in this
trial, will you approve us?

00:34:09.735 --> 00:34:10.710
It's impossible Question.

00:34:11.700 --> 00:34:13.110
Why ask that question.

00:34:13.980 --> 00:34:16.050
You're gonna get, they're
gonna say it's a review issue.

00:34:16.200 --> 00:34:17.550
We'll gather the information.

00:34:17.550 --> 00:34:18.720
It's a review issue.

00:34:18.840 --> 00:34:21.630
It's not a productive
question within that.

00:34:21.780 --> 00:34:25.170
And you know the answer they're
gonna give you think about how would

00:34:25.170 --> 00:34:27.420
you respond to these questions?

00:34:27.420 --> 00:34:30.420
And many of 'em, I don't know
how to respond to that, you

00:34:30.420 --> 00:34:31.770
know, but it's not productive.

00:34:31.770 --> 00:34:34.020
It doesn't move the ball
forward as part of that.

00:34:34.860 --> 00:34:40.080
Now statistics in in particular
there, it's not infrequently that.

00:34:40.935 --> 00:34:46.695
They've, uh, presented feedback
on the design within that.

00:34:47.235 --> 00:34:52.575
Um, I think maybe we've gotten it
wrong a little bit in, in, in that now,

00:34:52.935 --> 00:34:58.425
to some extent, I think they've said
something wrong, but that's our fault.

00:34:58.725 --> 00:35:04.095
We've provided all the material and all
of that, and so there are cases where

00:35:04.095 --> 00:35:05.775
we wanna walk through the details.

00:35:06.495 --> 00:35:11.385
We didn't explain this very well, and the
reason why is you'd really like to get

00:35:11.385 --> 00:35:16.785
this part of the design approved, whether
it's an adaptation or that part of it, and

00:35:16.785 --> 00:35:23.085
you think that further discussion would
clarify the particular issue, uh, on it.

00:35:23.415 --> 00:35:26.715
And that's really the goal of
that discussion in that part

00:35:26.715 --> 00:35:28.095
of the, the, the presentation.

00:35:28.095 --> 00:35:32.655
And that may be a part where I'm
speaking about specifically about that.

00:35:33.480 --> 00:35:37.020
And, and in my mind, by the way,
I may think they said something

00:35:37.020 --> 00:35:41.250
wrong, but I can never say that
and I, I, I'll say more about it.

00:35:41.250 --> 00:35:49.440
So the point may be sort of stat related
in that now the, um, the important

00:35:49.440 --> 00:35:54.000
part if I'm in a scenario like that
is you've got to give the FDA an

00:35:54.000 --> 00:35:59.430
out and being right might be wrong.

00:36:01.770 --> 00:36:07.650
You know, stomping on somebody and saying,
you're stupid for not knowing this.

00:36:07.650 --> 00:36:09.930
This is obvious, this is this, this.

00:36:10.080 --> 00:36:14.370
What do you think you're going to get for
a response from them for, for a scenario?

00:36:14.370 --> 00:36:17.970
So let me give you an
example, years ago example.

00:36:18.240 --> 00:36:20.040
Uh, it's not part of.

00:36:20.520 --> 00:36:26.580
My discussion tomorrow, we, it was a
relatively bad disease, wasn't oncology,

00:36:26.580 --> 00:36:32.970
but feels like oncology a little
bit, where, um, mortality is a risk.

00:36:33.030 --> 00:36:35.220
Uh, it's not mortality's, not just a risk.

00:36:35.250 --> 00:36:40.200
This isn't intensive care where they
die within 30 days or they live,

00:36:40.440 --> 00:36:43.470
this is relatively a deadly disease.

00:36:43.680 --> 00:36:49.020
High risk of mortality, maybe
median survival, six to nine months.

00:36:50.235 --> 00:36:53.025
And, uh, in this scenario, a bad disease.

00:36:53.025 --> 00:36:59.115
We proposed an analysis at a randomized
trial with a Cox proportional hazard

00:36:59.115 --> 00:37:00.855
model as the primary analysis.

00:37:01.635 --> 00:37:05.175
The FDA in their written feedback said no.

00:37:05.175 --> 00:37:06.195
And again.

00:37:07.335 --> 00:37:08.445
What kind of, no.

00:37:08.445 --> 00:37:14.955
Sometimes they say you can do that, but
we'd prefer X and it's kind of, alright,

00:37:14.955 --> 00:37:16.605
do we do something different in this?

00:37:16.605 --> 00:37:21.075
But they said, we want
a landmark analysis.

00:37:22.155 --> 00:37:26.925
What that means is, and I think they said
six months, we wanna know the proportion

00:37:26.925 --> 00:37:29.205
of patients that are alive at six months

00:37:31.275 --> 00:37:31.815
now.

00:37:31.935 --> 00:37:35.445
I think that's a bad analysis in this.

00:37:35.565 --> 00:37:36.195
Um.

00:37:36.645 --> 00:37:38.355
And it's inappropriate.

00:37:38.535 --> 00:37:43.275
By the way, oncology trials have this
constant risk of mortality and all that.

00:37:43.275 --> 00:37:46.125
They don't do landmark
analysis at one year.

00:37:46.125 --> 00:37:49.935
They summarize that, but that's not
the primary analysis of these trials.

00:37:50.685 --> 00:37:57.165
So the huge part is I want to get
them to accept a Cox model, a hazard

00:37:57.165 --> 00:38:01.665
ratio, instead of a landmark analysis.

00:38:02.415 --> 00:38:02.805
Now.

00:38:03.360 --> 00:38:05.910
The Cox model is gonna be more
powerful, but I think it's

00:38:05.910 --> 00:38:07.320
more clinically appropriate.

00:38:07.320 --> 00:38:09.030
It's more appropriate for the drug.

00:38:09.060 --> 00:38:13.890
And so now I'm presenting this and if
I come across as this is just stupid

00:38:13.890 --> 00:38:20.610
oncology's figured this out, you know, the
person can largely on the other end draw

00:38:20.610 --> 00:38:22.560
this and say, you know, I'm the regulator.

00:38:23.160 --> 00:38:26.190
And you've, you've sort of
alienated them at this point.

00:38:26.190 --> 00:38:31.020
You haven't given them an
out, and I'm the regulator.

00:38:31.020 --> 00:38:32.070
This is the decision.

00:38:32.925 --> 00:38:34.755
And you're kind of done at that point.

00:38:35.895 --> 00:38:42.405
So instead, you can walk through,
show the differences on, and, and, um,

00:38:42.525 --> 00:38:45.585
acknowledge why the person did that.

00:38:46.260 --> 00:38:50.820
And largely I think they do that
because it's a really nice summary.

00:38:51.210 --> 00:38:55.380
The knowing that 70% of people
are alive at six months relative

00:38:55.380 --> 00:39:00.270
to 40% alive at six months is a
pretty easy thing to understand.

00:39:00.270 --> 00:39:03.160
The Sta, there aren't a lot of
statistical assumptions to that.

00:39:03.675 --> 00:39:04.935
Very straightforward.

00:39:05.565 --> 00:39:07.935
Now, the challenge here
is, is it six months?

00:39:07.935 --> 00:39:08.775
Is it nine months?

00:39:08.775 --> 00:39:09.405
Is it 12 months?

00:39:09.405 --> 00:39:12.975
And so you can sort of walk through
this and what we, we entertain

00:39:12.975 --> 00:39:14.595
doing these landmark analysis.

00:39:14.595 --> 00:39:18.495
We think we, we may even push
patients in the right time to do it.

00:39:18.495 --> 00:39:20.325
Might be 12 months, it
might be six months.

00:39:20.625 --> 00:39:22.935
The Cox models really
integrating all of these.

00:39:23.390 --> 00:39:25.430
So you've, you've represented to them.

00:39:25.460 --> 00:39:28.250
You understand why they
came back with this comment.

00:39:28.430 --> 00:39:30.170
Here's why we'd like to do it.

00:39:30.170 --> 00:39:34.820
By the way, we're gonna show you
the six month rate and that'll be a

00:39:34.820 --> 00:39:37.070
secondary endpoint and all of that.

00:39:37.070 --> 00:39:38.930
Do you think that's an
appropriate way forward?

00:39:39.770 --> 00:39:43.460
So you've, you've, you've
respected what they did.

00:39:43.460 --> 00:39:45.830
You've acknowledged why they say that.

00:39:46.130 --> 00:39:50.240
You've presented why you think your
way is better in a way that they

00:39:50.240 --> 00:39:52.190
can say yes to that and it's not.

00:39:52.740 --> 00:39:53.940
You were right and I was wrong.

00:39:54.660 --> 00:39:58.080
You don't want a yes to be an
acknowledgement that they were

00:39:58.080 --> 00:40:00.990
wrong and you were right, because
then you're not gonna get a yes.

00:40:01.620 --> 00:40:04.020
And that's a lot of times
what I'm thinking about.

00:40:04.020 --> 00:40:08.970
And the, the challenging part of
this, uh, uh, is it, and it's always

00:40:08.970 --> 00:40:10.860
sort of the challenging part of that.

00:40:11.130 --> 00:40:15.720
So allowing a yes to be had in that.

00:40:15.720 --> 00:40:17.610
And sometimes it's not even a yes.

00:40:17.610 --> 00:40:18.630
It's not a no.

00:40:19.980 --> 00:40:20.220
Uh.

00:40:20.820 --> 00:40:24.960
I under and, and if they say, I
understand your argument, we're gonna be

00:40:24.960 --> 00:40:27.000
particularly interested in six months.

00:40:27.000 --> 00:40:31.560
We think you should do six months,
but you can do a Cox mom, that

00:40:31.560 --> 00:40:33.330
might be a great outcome for you.

00:40:34.185 --> 00:40:37.485
And it's not a no, and,
and and all of that.

00:40:37.485 --> 00:40:41.115
So again, that's the part
of this, um, uh, within it.

00:40:41.115 --> 00:40:43.995
And there are some times where, you
know, you're never gonna get them to

00:40:43.995 --> 00:40:49.635
say yes, but you're trying to get them
not to say no in a particular scenario,

00:40:50.020 --> 00:40:52.125
uh, uh, for the particular design.

00:40:52.125 --> 00:40:58.215
You gotta give 'em that out in a way
that is, is, uh, allows respect and,

00:40:58.215 --> 00:41:00.520
and, and recognizing, uh, of that.

00:41:01.905 --> 00:41:04.515
Um, and that can be a challenge for me.

00:41:05.115 --> 00:41:09.255
Sometimes what gets written in there
is something statistically that

00:41:09.255 --> 00:41:16.485
I think is wrong and I, I wanna
win that battle and, you know,

00:41:16.485 --> 00:41:19.275
that's not the point to it, to it.

00:41:19.365 --> 00:41:24.675
Um, you know, I can be wrong by trying
to show that I'm right in that scenario.

00:41:24.675 --> 00:41:26.745
It, that's how productive
it doesn't move the ball.

00:41:27.390 --> 00:41:30.930
Written feedback is
much easier to provide.

00:41:30.930 --> 00:41:33.210
Something is not a game changer.

00:41:33.390 --> 00:41:38.070
It doesn't alter where you're going,
but you think they haven't understood

00:41:38.070 --> 00:41:41.370
why your information fraction
for a group sequential design is

00:41:41.370 --> 00:41:43.230
appropriate or, or, or something else.

00:41:43.530 --> 00:41:45.330
It written feedback.

00:41:45.570 --> 00:41:50.880
Is enough for that and it's easier to
accept things or research things or wi wi

00:41:50.880 --> 00:41:54.750
within that, but it might not be something
critical to where it goes within that.

00:41:54.750 --> 00:41:58.140
So written feedback as a
statistician to things you think

00:41:58.140 --> 00:42:02.520
are wrong can be much better than
a discussion in front of everybody.

00:42:03.180 --> 00:42:08.580
Many times I'm presenting
statistical stuff and I'm trying

00:42:08.580 --> 00:42:13.140
to appeal to the clinical team
as much as I am the statistician.

00:42:14.310 --> 00:42:19.020
If this is a scenario of Bayesian
borrowing of an alternative data

00:42:19.020 --> 00:42:24.180
set, part of it is the mechanism,
the Bayesian hierarchical model that

00:42:24.180 --> 00:42:29.010
I'm presenting, but I'm really trying
to appeal to the clinician why this

00:42:29.010 --> 00:42:34.680
is clinically appropriate, how it
behaves, what the estimate's gonna

00:42:34.680 --> 00:42:38.615
look like, what's the confidence
interval in a way that they see it.

00:42:39.240 --> 00:42:42.870
And, and that even though I'm
the statistician, I'm kind of

00:42:42.870 --> 00:42:48.570
presenting to somebody else is,
is a challenge that I'm frequently

00:42:48.570 --> 00:42:51.270
dealing with, uh, in these scenarios.

00:42:52.200 --> 00:42:58.740
Um, I, I had an experience one time
with Roger Lewis who works here at

00:42:58.740 --> 00:43:02.040
Berry Consultants, a brilliant, uh, um.

00:43:02.385 --> 00:43:06.405
Clinician, he's an emergency room
doctor, but a, a brilliant guy.

00:43:06.795 --> 00:43:12.405
Um, I was sitting next to him at one of
these FDA meetings and somebody from the

00:43:12.405 --> 00:43:17.535
FDA said something that I thought was just
wrong, and I thought we had a really good

00:43:17.535 --> 00:43:22.845
argument and I was just so excited to jump
up and say, and, and I started to respond.

00:43:23.445 --> 00:43:25.785
Meanwhile, Bob Temple.

00:43:27.315 --> 00:43:31.845
Responded to the FDA person
and was making my point.

00:43:32.805 --> 00:43:37.755
Roger grabbed me and largely
really looked at me like, really,

00:43:38.025 --> 00:43:40.545
Bob Temple is making your point.

00:43:40.725 --> 00:43:40.875
Shut up.

00:43:42.525 --> 00:43:43.275
You know, kind of thing.

00:43:43.275 --> 00:43:49.905
So, uh, and I did, I I took the hint,
I shut up and it's much better coming

00:43:49.905 --> 00:43:51.555
from, I didn't have to say anything.

00:43:51.765 --> 00:43:53.085
The point was made.

00:43:53.325 --> 00:43:58.215
It's very different when somebody
from the FDA presents that part of it,

00:43:58.215 --> 00:44:01.245
and not, and not me, uh, within that.

00:44:01.245 --> 00:44:04.545
But, so I can sometimes get
excited about trying to be right.

00:44:04.875 --> 00:44:08.235
And remember, it's, it's all
part of a bigger battle, uh,

00:44:08.235 --> 00:44:09.265
of what we're trying to do.

00:44:10.485 --> 00:44:18.705
Um, uh, within that, I don't get,
I, i i a last resort for me is

00:44:18.705 --> 00:44:25.245
appealing to guidances appealing to
a paper from a famous statistician

00:44:25.245 --> 00:44:27.555
that says, A is better than B.

00:44:28.215 --> 00:44:28.755
Within that.

00:44:30.135 --> 00:44:34.275
I, I, you don't get very far with
that, and I feel it's almost a

00:44:34.275 --> 00:44:41.835
last resort to say the own, your
own FDA guidance says X, Y, Z.

00:44:42.075 --> 00:44:49.425
It's almost bringing legal standards
into it, and at that point it's somewhat.

00:44:49.440 --> 00:44:53.160
I, I, I don't think it's productive now.

00:44:53.160 --> 00:44:57.690
There, there can be time and place
for it, and I know that can be part

00:44:57.690 --> 00:45:03.270
of a written response as we think
this is also supported by this, but

00:45:03.270 --> 00:45:07.800
trying to use that to win an argument,
I, I, I've just never been very

00:45:07.800 --> 00:45:10.110
successful, uh, at that part of it.

00:45:10.110 --> 00:45:11.190
Now I kind of live.

00:45:11.780 --> 00:45:14.750
In many different spaces
than, than others.

00:45:15.020 --> 00:45:19.190
And it depends on what you're discussing
within that, you know, if it's a guidance

00:45:19.190 --> 00:45:23.030
on secondary endpoints and the way
something is done, and it's relatively,

00:45:23.420 --> 00:45:27.620
it's a relatively non, uh, critical part.

00:45:27.620 --> 00:45:28.040
Yeah.

00:45:28.160 --> 00:45:29.120
Uh, absolutely.

00:45:29.120 --> 00:45:32.690
We're, you know, we, we tried to follow
this guidance sort of thing, but to

00:45:32.690 --> 00:45:36.650
prove a point, pulling out guidances
haven't been successful at me, so I.

00:45:37.440 --> 00:45:39.000
I try never to do that.

00:45:39.240 --> 00:45:44.070
I feel like once you break out the legal
stuff, this is the, the productivity

00:45:44.070 --> 00:45:45.660
of the meeting is kind of shot.

00:45:47.820 --> 00:45:48.450
Okay.

00:45:49.350 --> 00:45:55.020
So at the end of the meeting, the
hour is up and you're, you're.

00:45:55.455 --> 00:46:01.935
Project manager from the FDA reminds
anything we're expecting X from the this.

00:46:01.935 --> 00:46:06.015
You'll hear this and it's usually
we're gonna get the minutes finalized,

00:46:06.015 --> 00:46:11.145
you know this, but very, uh, uh,
sort of process part of this.

00:46:11.145 --> 00:46:16.605
And they take you out and the meeting's
over, uh, and typically everybody runs to

00:46:16.605 --> 00:46:20.775
the airport at that point, or you may have
a, a post meeting within a, any of that.

00:46:21.285 --> 00:46:21.765
So.

00:46:23.190 --> 00:46:25.080
That'll be my day tomorrow.

00:46:25.710 --> 00:46:28.530
Uh, within this, I'm very
much looking forward to it.

00:46:28.605 --> 00:46:31.140
I, I always look forward
to these meetings.

00:46:31.140 --> 00:46:34.500
I, I think it's really,
uh, having an impact.

00:46:34.500 --> 00:46:41.370
So I, I, I love these meetings and again,
I, I, I, because I truly respect what the,

00:46:41.370 --> 00:46:45.990
the people at the FDA are doing and their
role they play in, in all of this process.

00:46:45.990 --> 00:46:47.965
So, uh, this will be exciting.

00:46:49.560 --> 00:46:51.930
So thanks everybody for joining me.

00:46:52.320 --> 00:46:54.330
Uh, wish me luck tomorrow.

00:46:54.540 --> 00:46:56.340
Wish science luck tomorrow.

00:46:56.670 --> 00:47:01.680
Uh, and until next time, we'll
be right here in the interim,