WEBVTT

NOTE
This file was generated by Descript 

00:00:00.000 --> 00:00:03.119
Judith: Welcome to Berry's In the
Interim podcast, where we explore the

00:00:03.119 --> 00:00:07.469
cutting edge of innovative clinical
trial design for the pharmaceutical and

00:00:07.469 --> 00:00:09.880
medical industries, and so much more.

00:00:10.359 --> 00:00:11.039
Let's dive in.

00:00:12.561 --> 00:00:14.001
Scott Berry: Welcome everybody.

00:00:14.001 --> 00:00:17.901
Back to, in the interim,
I'm Scott Berry, your host.

00:00:18.801 --> 00:00:23.421
So I, for those of you who join
us in the interim frequently, you

00:00:23.421 --> 00:00:28.311
may notice, uh, my background is
different than our usual studio.

00:00:28.821 --> 00:00:34.071
Uh, it's, uh, stunningly beautiful
place and I will, I will lift the

00:00:34.071 --> 00:00:38.991
camera up a little bit and show that
I'm actually in Southern California.

00:00:40.251 --> 00:00:44.661
Uh, taping here next to the
grand old Pacific Ocean.

00:00:44.661 --> 00:00:45.711
And it's beautiful.

00:00:46.581 --> 00:00:48.291
Um, or am I,

00:00:50.391 --> 00:00:56.961
and, and many of you who join, uh,
uh, webinars, uh, zoom meetings,

00:00:57.621 --> 00:01:02.031
lots of people have virtual
backgrounds and a lot of people

00:01:02.031 --> 00:01:03.471
have the same virtual background.

00:01:03.471 --> 00:01:05.691
Makes 'em look like
they're in the same office.

00:01:05.811 --> 00:01:07.016
Uh, so you don't know.

00:01:08.121 --> 00:01:14.211
If I'm actually on the Pacific Ocean
or it's virtual digital and I am stuck

00:01:14.211 --> 00:01:21.081
in the bowels of my attic in Austin,
Texas, or I really am on the coast,

00:01:22.761 --> 00:01:27.021
so by the way, I am of course, am I?

00:01:27.021 --> 00:01:30.381
Am I Scott Berry or am I a
digital twin of Scott Berry?

00:01:31.416 --> 00:01:36.156
So that's my topic for today
are, are digital twins.

00:01:36.156 --> 00:01:41.346
And it, it's been a term
used, uh, in clinical trials.

00:01:41.346 --> 00:01:43.176
It plays a role in clinical trials.

00:01:43.176 --> 00:01:44.496
It plays a role in ai.

00:01:44.496 --> 00:01:47.856
What is it besides perhaps.

00:01:48.276 --> 00:01:52.776
A picture of the, the, the Pacific
Ocean, which is not a digital twin.

00:01:52.776 --> 00:01:57.246
A a picture of something is not
a digital twin, but let's talk

00:01:57.246 --> 00:02:02.106
about what it is, how they're used
in trials, the potential of them.

00:02:02.406 --> 00:02:05.676
And my viewpoint, I've actually asked
quite a bit within trials, we do a

00:02:05.676 --> 00:02:11.796
lot of adaptive trials, hence sitting
in the interim of, of a clinical

00:02:11.796 --> 00:02:13.901
trial, uh, right now where you are.

00:02:14.601 --> 00:02:18.831
Or, and we do a lot of Bayesian
analyses, which can be separate

00:02:18.831 --> 00:02:20.301
from using external data.

00:02:20.511 --> 00:02:25.281
You can do Bayesian analyses without
using external data, but when you use

00:02:25.281 --> 00:02:29.571
external data to a trial, the Bayesian
approach is a really powerful thing.

00:02:29.571 --> 00:02:33.261
So we end up doing quite a
bit of trials using external.

00:02:33.621 --> 00:02:38.451
Yeah, digital twins are external in
some sense, to the clinical trial.

00:02:38.451 --> 00:02:43.071
So I'm asked a lot about digital twins
and so I, I, I will give you my viewpoint

00:02:43.071 --> 00:02:46.371
today here, uh, uh, in the interim.

00:02:47.481 --> 00:02:49.521
So the name digital twin.

00:02:50.781 --> 00:02:55.131
Uh, is, is I, I'll tell
you what Wikipedia says.

00:02:55.341 --> 00:03:01.941
A digital twin is a digital model
of an intended or actual real world

00:03:01.941 --> 00:03:05.541
physical product system or process.

00:03:06.246 --> 00:03:07.506
A physical twin.

00:03:07.506 --> 00:03:12.156
That's the physical thing that
we're actually interested in.

00:03:12.606 --> 00:03:18.726
It serves as the effectively
indistinguishable digital counterpart

00:03:19.416 --> 00:03:24.096
of it for practical purposes,
such as simulation, integration,

00:03:24.096 --> 00:03:25.026
testing, monitoring and maintenance.

00:03:27.571 --> 00:03:30.846
So a digital twin is a.

00:03:31.776 --> 00:03:38.256
A creation a, a model-based
creation, a simulation of a real

00:03:38.256 --> 00:03:40.386
physical thing that's of interest.

00:03:40.986 --> 00:03:48.456
The original term traces back to
NASA, where they're creating digital.

00:03:48.936 --> 00:03:54.096
Simulations digital twins of
a physical characteristic.

00:03:54.126 --> 00:03:58.476
This could be a rocket ship,
this could be a new airplane.

00:03:58.776 --> 00:04:04.116
This could be many different, uh,
interesting physical things that

00:04:04.116 --> 00:04:06.126
we can't do a ton of testing on.

00:04:06.456 --> 00:04:11.526
You could imagine the Department
of Energy doing this for, uh, uh,

00:04:11.556 --> 00:04:13.686
tests of nuclear testing where.

00:04:13.861 --> 00:04:16.111
We can't just be blowing things up.

00:04:16.111 --> 00:04:21.241
So we're creating digital
twins of that physical system.

00:04:22.741 --> 00:04:27.661
Incredibly powerful, but of course
powerful to the extent of its

00:04:27.661 --> 00:04:30.391
realism and in in this scenario.

00:04:30.391 --> 00:04:30.451
So.

00:04:31.836 --> 00:04:35.286
Let me give you an example of a
digital twin that you, we've never

00:04:35.286 --> 00:04:40.356
called a digital twin before, but
it is completely a digital twin, and

00:04:40.356 --> 00:04:44.436
it's something we do quite a bit of,
which is clinical trial simulation.

00:04:44.886 --> 00:04:51.096
The, the Wikipedia definition of a
digital twin talks about simulation.

00:04:51.816 --> 00:05:00.216
We, in the design of clinical trials,
we create an silico digital version.

00:05:00.581 --> 00:05:02.411
Of the physical trial.

00:05:02.621 --> 00:05:05.291
The physical trial is run once.

00:05:06.671 --> 00:05:13.301
We don't want to go run 10,000
physical trials to figure out the

00:05:13.301 --> 00:05:15.911
best way to design that trial.

00:05:16.181 --> 00:05:22.871
What's a good trial that would be, you
know, many, many years we can't do it.

00:05:23.141 --> 00:05:28.601
So we create a clinical trial
simulation of that physical trial.

00:05:29.871 --> 00:05:36.291
We try to create it with an,
with all of the needed detail of

00:05:36.291 --> 00:05:38.181
the things we're going to vary.

00:05:39.591 --> 00:05:44.391
So we, we, we do enrollment rates,
variability to enrollment rates.

00:05:44.391 --> 00:05:49.341
Typically a s on process of that, where
there, there's ramping up of sites.

00:05:49.611 --> 00:05:54.921
We incorporate dropouts of patients, the
behavior of patients on the controls.

00:05:55.251 --> 00:06:00.111
Maybe they're seeing regression to
the mean within that scenario, then we

00:06:00.111 --> 00:06:05.541
simulate their outcomes in the trial
with variabilities to their outcomes.

00:06:05.541 --> 00:06:07.716
We even test treatment effects in that.

00:06:09.381 --> 00:06:20.121
So we create this digital system that
has value to us as a digital twin of that

00:06:20.121 --> 00:06:27.381
physical system, of that single physical
trial that we run once the value of this

00:06:27.381 --> 00:06:31.731
is now in silico, this, this simulation.

00:06:32.091 --> 00:06:33.981
We can do things to that.

00:06:34.041 --> 00:06:38.031
We can add response,
adaptive randomization to it.

00:06:38.031 --> 00:06:41.691
What if we change the design to have that?

00:06:42.141 --> 00:06:47.091
Then we run it tens of thousands,
millions of times, and we

00:06:47.091 --> 00:06:48.621
look at how well does it work.

00:06:49.551 --> 00:06:53.241
Does it get the right answer where
we feed in the right answer to it?

00:06:53.541 --> 00:06:58.161
This might be very similar to what
NASA does, where they're simulating

00:06:58.581 --> 00:07:03.411
different aerodynamics to the systems,
different weights to the system.

00:07:03.621 --> 00:07:06.051
What if we alter the wing shape to it?

00:07:06.051 --> 00:07:07.611
How is it going to behave?

00:07:07.821 --> 00:07:10.071
What if we change the material to it?

00:07:10.461 --> 00:07:14.091
That's what we're doing when we're
doing clinical trial simulation.

00:07:14.811 --> 00:07:15.801
We vary it.

00:07:15.801 --> 00:07:17.451
We vary the design.

00:07:17.696 --> 00:07:20.186
And we look at how well does it fly?

00:07:20.636 --> 00:07:23.156
Does, is it, is it
efficient in its flight?

00:07:23.156 --> 00:07:24.926
Does it get to the right answer?

00:07:25.136 --> 00:07:27.956
Where the beauty of it is, we're
pumping in the right answer.

00:07:27.956 --> 00:07:31.406
At the same time, we, we get to play
the role of Mother Nature and say,

00:07:31.406 --> 00:07:34.706
suppose the treatment has a 50% effect.

00:07:35.636 --> 00:07:40.256
Does the trial, the clinical trial
simulation get the right answer?

00:07:41.906 --> 00:07:44.666
Now, there are critics of this.

00:07:45.816 --> 00:07:50.286
Um, and they're, they're critics
of clinical trial simulation.

00:07:50.526 --> 00:07:56.106
And of course the clinical trial
simulation is only as valuable as how it

00:07:56.106 --> 00:07:59.436
reproduces reality of the physical system.

00:08:00.486 --> 00:08:05.256
Now, usually when we're doing it,
we put in the detail needed to

00:08:05.256 --> 00:08:07.296
test the things we're gonna test.

00:08:08.241 --> 00:08:15.561
We may not put in there detailed aspects
of the biology of a patient, because

00:08:15.561 --> 00:08:17.301
that's not what we're interested in.

00:08:17.301 --> 00:08:24.291
We're not actually PKPD do digital
twin simulation of the human body.

00:08:24.681 --> 00:08:29.451
What happens when we give this drug
and it's half life is a certain thing.

00:08:29.451 --> 00:08:31.551
What happens to the physical system?

00:08:31.881 --> 00:08:33.891
And how do they forecast outcomes?

00:08:33.891 --> 00:08:35.061
It's a beautiful thing.

00:08:35.211 --> 00:08:36.651
That's not my specialty.

00:08:36.651 --> 00:08:38.061
I I, I don't do that.

00:08:38.691 --> 00:08:40.131
But we do it in a clinical trial.

00:08:40.131 --> 00:08:41.751
We do it to the detailed needed.

00:08:42.261 --> 00:08:47.151
So there are critics of simulation and
in some ways, if you could analytically

00:08:47.151 --> 00:08:52.551
calculate all these things, you don't
need simulation when doing a simple

00:08:52.551 --> 00:08:56.751
fixed trial of a single endpoint.

00:08:56.751 --> 00:08:58.371
And you're interested in the power.

00:08:58.371 --> 00:09:00.296
We, we can go look this up in tables.

00:09:01.491 --> 00:09:03.321
We, we have simple formulas.

00:09:03.321 --> 00:09:09.081
We have analytical characterization
of that physical system that

00:09:09.081 --> 00:09:10.371
you don't need simulation.

00:09:11.151 --> 00:09:15.651
And you can ma, you can imagine
an absolute brilliant person

00:09:16.191 --> 00:09:20.841
that can do analytical work
way beyond my capabilities.

00:09:21.291 --> 00:09:22.941
Say We don't need simulation.

00:09:22.941 --> 00:09:24.471
I can do this analytically.

00:09:25.236 --> 00:09:29.646
Now I think that only works to an
extent of the complexity of your

00:09:29.646 --> 00:09:33.516
design, and it can be limiting
in the complexity of your design.

00:09:33.936 --> 00:09:39.216
If you believe we have to analytically
understand the trial, it immediately

00:09:39.216 --> 00:09:43.896
sets the complexity to, to a level
that I, I, I think is limiting.

00:09:44.166 --> 00:09:49.686
Okay, but we want to understand this,
where we understand the role of dropouts

00:09:49.986 --> 00:09:52.866
add really more complex designs.

00:09:53.196 --> 00:09:54.996
10, 15 analyses.

00:09:54.996 --> 00:09:58.086
We're dropping patient
character, patient types.

00:09:58.086 --> 00:09:59.826
We're changing randomization.

00:10:00.066 --> 00:10:01.986
We have multiple arms in the trial.

00:10:02.691 --> 00:10:05.421
To a complexity that you
can't do it analytically.

00:10:05.421 --> 00:10:09.981
And then this is a hugely valuable
thing to have this digital

00:10:09.981 --> 00:10:11.811
twin of the physical system.

00:10:12.141 --> 00:10:14.541
And then we alter it and
we create a better design.

00:10:16.431 --> 00:10:21.441
So this little rub of analytically
and simulation is going to to to,

00:10:21.471 --> 00:10:23.541
to play a role in multiple ways.

00:10:23.541 --> 00:10:30.501
Anytime somebody creates a digital
twin, there's going to be this rub now.

00:10:30.876 --> 00:10:36.876
Digital twins aren't brought up
as clinical trial simulation in

00:10:36.876 --> 00:10:41.736
the world of clinical trials, but
it is absolutely a digital twin.

00:10:42.696 --> 00:10:45.756
Digital twins are a term
that a company is using.

00:10:45.756 --> 00:10:47.706
The company's name is unlearn.

00:10:48.706 --> 00:10:52.881
I, I, I, I don't have any
relationship to any to, to unlearn.

00:10:52.886 --> 00:10:53.991
I, I, I've talked to them.

00:10:53.991 --> 00:10:57.351
I think they're, they're, they're
quite brilliant and they've, they've

00:10:57.351 --> 00:11:01.221
created something and it, it may
be even some level of proprietary

00:11:01.581 --> 00:11:06.411
where they label things as a digital
twin, and that's been making its way.

00:11:06.621 --> 00:11:10.671
But this whole concept is
also really a quite powerful

00:11:10.671 --> 00:11:12.051
thing within clinical trials.

00:11:13.341 --> 00:11:17.961
The use of it in clinical trials
has usually been about a patient.

00:11:18.951 --> 00:11:25.011
Can we create a digital twin of a
patient, not to a biologic level of

00:11:25.011 --> 00:11:29.001
what are the level of cholesterol,
and then giving them a statin?

00:11:29.001 --> 00:11:35.241
What happens biologically, but one
where we create a digital twin of

00:11:35.241 --> 00:11:40.486
what their endpoint would be in
a trial or endpoints of interest.

00:11:41.166 --> 00:11:47.676
If they were on a control, if they
behave in a way that we have previous

00:11:47.676 --> 00:11:52.866
data, that we can create a potential
digital twin of that patient.

00:11:53.946 --> 00:11:58.536
So, uh, we'll use as an example, a
couple examples, and I think unlearn

00:11:58.596 --> 00:12:02.076
talks about Alzheimer's and a LS.

00:12:02.841 --> 00:12:06.681
And so we have a patient that
comes in in a clinical trial.

00:12:06.891 --> 00:12:10.911
They have a baseline CDR sum of boxes.

00:12:10.911 --> 00:12:17.451
They have a baseline amyloid level, they
have baseline MMSE level, and we're gonna

00:12:17.451 --> 00:12:22.731
run a trial where we're going to look
at 18 month outcomes of that patient.

00:12:22.731 --> 00:12:26.391
And we're interested in what is
that patient's CDR sum of boxes.

00:12:26.751 --> 00:12:28.701
This is a clinical dimensional.

00:12:29.076 --> 00:12:34.986
Clinical dementia rating scale endpoint
commonly used in Alzheimer's trials.

00:12:35.286 --> 00:12:40.146
And we would like to understand what is
going to happen to that patient if they

00:12:40.146 --> 00:12:44.496
were given a placebo 18 months from now.

00:12:44.586 --> 00:12:47.136
What is their CDR sum
of boxes going to be.

00:12:49.356 --> 00:12:51.726
Incredibly powerful thing.

00:12:52.091 --> 00:12:55.211
Uh, if, if we can do this now.

00:12:56.421 --> 00:12:58.911
Think about why we run a clinical trial.

00:12:59.511 --> 00:13:06.951
We run clinical trials where we give a
treatment to a patient, and what we really

00:13:06.951 --> 00:13:10.641
want to know is the counterfactual of
what would've happened to that patient

00:13:10.641 --> 00:13:17.091
Jane Doe if we didn't give them the
treatment in a, in an Alzheimer's trial.

00:13:17.091 --> 00:13:22.401
Their CDR sum of boxes goes
from one to four in the trial.

00:13:23.241 --> 00:13:25.491
What would've happened if we
didn't give them the treatment?

00:13:25.521 --> 00:13:29.331
Would it have gone one to seven,
higher values are worse on that input?

00:13:29.811 --> 00:13:30.861
Would they have been worse?

00:13:30.861 --> 00:13:31.731
Would they have been better?

00:13:31.731 --> 00:13:33.591
Would they have been exactly the same?

00:13:34.161 --> 00:13:35.601
That's why we run a trial.

00:13:36.171 --> 00:13:40.311
It's why we randomized patients
in a trial to no treatment.

00:13:41.076 --> 00:13:46.086
So that we see some patients under the
counterfactual, uh, they're, they're

00:13:46.086 --> 00:13:49.716
not actually the counterfactual, they're
the factual of giving them a placebo.

00:13:49.866 --> 00:13:53.616
We, we observe what happens under
placebo and what happens under treatment.

00:13:54.366 --> 00:14:02.436
Imagine the power that we could
give 500 patients a treatment.

00:14:03.531 --> 00:14:09.471
And we could just say, what is the digital
twin of that patient from 18 months?

00:14:09.921 --> 00:14:11.391
And we analyzed the trial.

00:14:11.391 --> 00:14:14.031
I'll talk a little bit more about
how, how, how would you do that?

00:14:14.211 --> 00:14:15.201
But we analyzed the trial.

00:14:15.201 --> 00:14:20.751
So comparing to that, our trials
can be less than half the size

00:14:20.751 --> 00:14:23.781
actually, because there's no,
there's no variability to that.

00:14:23.811 --> 00:14:25.281
Maybe we need to characterize that.

00:14:26.001 --> 00:14:31.036
But imagine if you could say, Hey Alexa,
what would've happened to Jane Doe?

00:14:32.451 --> 00:14:37.551
Without this experimental
treatment, incredibly powerful.

00:14:38.631 --> 00:14:43.131
It's why we randomize, and
it's why we do clinical trials.

00:14:43.131 --> 00:14:44.661
It's why we do blinding.

00:14:44.661 --> 00:14:50.601
It's why we do all of the scientific
aspects of these trials, and it's why

00:14:50.601 --> 00:14:57.501
they're quite large, quite expensive,
is to get that honest answer to it Now.

00:14:58.191 --> 00:15:03.081
Let's revisit a little bit of what
Unlearned does, because I think it's

00:15:03.081 --> 00:15:07.461
important because they have come out
with an announcement and you can go

00:15:07.491 --> 00:15:12.441
look at unlearn and you can look at
their news where the, the, the title

00:15:12.441 --> 00:15:17.841
of their press release is that the
European Medicines Agency qualifies

00:15:17.931 --> 00:15:25.401
unlearned AI powered method for running
smaller, faster clinical trials.

00:15:26.031 --> 00:15:27.486
So the immediate thing is, oh wow.

00:15:28.086 --> 00:15:34.476
We're doing this, what, what, what Scott
said about, uh, uh, creating a digital

00:15:34.476 --> 00:15:40.056
twin and we're comparing it to the
digital twin and EMA has qualified this,

00:15:40.596 --> 00:15:42.066
so I think you have to take a step back.

00:15:42.066 --> 00:15:44.826
So what, what, what have they qualified?

00:15:45.186 --> 00:15:49.686
Uh, of course there's this comment of AI
powered, we'll, we'll come back to that.

00:15:49.686 --> 00:15:54.336
This AI powered method for running
smaller, faster clinical trials.

00:15:55.731 --> 00:16:03.591
So, and you can go read this on Unlearned,
but what Unlearned does is they have

00:16:03.591 --> 00:16:08.961
data, they have historical data on
Alzheimer's or a LS or the disease.

00:16:08.961 --> 00:16:14.451
You need it where they
create in AI based model.

00:16:14.451 --> 00:16:18.501
And it says on their website that
they use deep neural networks.

00:16:20.391 --> 00:16:24.921
So this is a little bit about trying
to get into the details statistically.

00:16:24.921 --> 00:16:33.351
We understand a little bit about what
this means to predict or model digitally.

00:16:33.531 --> 00:16:36.771
What would happen to a
patient 18 months from now?

00:16:38.136 --> 00:16:42.216
So they create this neural network model.

00:16:42.216 --> 00:16:46.296
They call it an AI model, but
it's a, it's a neural network, and

00:16:46.296 --> 00:16:47.736
it's not just a neural network.

00:16:47.736 --> 00:16:52.056
It's a deep neural network
to do this forecast.

00:16:52.866 --> 00:16:59.886
Their forecast is not a single patient
forecast with variability to it.

00:17:00.216 --> 00:17:03.006
It's essentially a model mean prediction.

00:17:03.801 --> 00:17:08.811
It's a model average prediction
of a neural network based on their

00:17:08.811 --> 00:17:16.701
data of what that patient would do
with a LS, with Alzheimer's, with

00:17:16.821 --> 00:17:20.901
cardiovascular disease, with weight loss,
whatever it is they're trying to do.

00:17:21.321 --> 00:17:22.371
That's what it is.

00:17:22.371 --> 00:17:24.411
It's a single model prediction.

00:17:24.951 --> 00:17:30.106
Now, what has EMA qualified,
it's not that that.

00:17:30.861 --> 00:17:36.711
Digital twin is a comparison
to a patient on treatment.

00:17:37.191 --> 00:17:40.341
They use it in what they call pro cova.

00:17:40.641 --> 00:17:42.771
It's a covariate adjustment.

00:17:43.941 --> 00:17:50.031
Now, in in modeling, you've
probably heard of an ancova where

00:17:50.031 --> 00:17:52.791
we use a baseline covariate.

00:17:53.286 --> 00:17:59.166
Baseline CDR sum of boxes to
adjust for the 18 month value.

00:17:59.166 --> 00:18:00.186
That's an ancova.

00:18:00.396 --> 00:18:06.756
So this is a pro cova, and this is
that you use the predicted value

00:18:07.656 --> 00:18:10.536
of 18 months for that digital twin.

00:18:10.746 --> 00:18:14.796
You put that in as a covariate
to the model, and all of that

00:18:14.796 --> 00:18:16.566
is based on baseline things.

00:18:16.776 --> 00:18:18.606
Very legitimate thing to do.

00:18:19.266 --> 00:18:21.396
And that's your covariate.

00:18:21.396 --> 00:18:27.816
So this deep neural network value based on
the historical data is your covariate and

00:18:27.816 --> 00:18:30.126
you put it in for treatment and control.

00:18:30.696 --> 00:18:35.436
It's, it's, think of it as a value
of p this prediction for every

00:18:35.436 --> 00:18:40.056
patient, they, they put it in
the, the, this AI powered thing.

00:18:40.056 --> 00:18:44.496
And what comes out of it is
an 18 month value of 3.7,

00:18:45.426 --> 00:18:47.791
and that goes in as a
covariate in your model.

00:18:49.506 --> 00:18:52.866
And this creates smaller,
more efficient trials.

00:18:54.426 --> 00:18:56.466
So what, what are the benefits here?

00:18:56.466 --> 00:18:56.916
Now what?

00:18:56.916 --> 00:18:58.746
What's an alternative to this?

00:18:59.196 --> 00:19:06.126
An alternative to that is that you put
in the baseline value, you put in their

00:19:06.126 --> 00:19:11.856
amyloid value, you put in their MMSE,
you put in characteristics of that

00:19:11.856 --> 00:19:16.626
patient as covariates in the model and.

00:19:17.061 --> 00:19:22.311
The, the trial fits those covariates
based on the patients in the trial

00:19:22.431 --> 00:19:27.471
in both arms and, and really that
covariate adjustment is providing a

00:19:27.471 --> 00:19:30.501
prediction of that patient at 18 months.

00:19:30.771 --> 00:19:35.901
That prediction balances out the
two treatment arms in the scenario.

00:19:37.041 --> 00:19:42.111
It allows us to reduce the variability
of this outcome at 18 months.

00:19:42.321 --> 00:19:47.631
If we don't do any covariate adjustment,
there's more variability, but by Covariate

00:19:47.631 --> 00:19:52.281
adjustment, we reduce that variability
substantially in many circumstances.

00:19:52.581 --> 00:19:58.251
Baseline CDR, sum of Boxes,
amyloid value, MMSE, Addis Cog

00:19:58.701 --> 00:20:02.421
plugged in is going to reduce the
variability of the 18 month outcome.

00:20:02.421 --> 00:20:03.651
Incredibly important.

00:20:04.761 --> 00:20:06.921
So now let's compare those two things.

00:20:06.921 --> 00:20:16.071
One trial uses baseline covariates
and does not use unlearned, uh uh, EMA

00:20:16.071 --> 00:20:18.771
qualified thing, just baseline covariates.

00:20:18.771 --> 00:20:20.061
It's doing this adjustment.

00:20:20.301 --> 00:20:24.801
There's gonna be a certain variability
to the 18 month outcome after adjustment.

00:20:25.611 --> 00:20:31.611
The other trial uses this pro
cova digital twin adjustment.

00:20:32.716 --> 00:20:35.781
Yeah, there's gonna be a certain
amount of 18 month variability.

00:20:35.781 --> 00:20:38.091
The role of both of these are identical.

00:20:38.091 --> 00:20:40.341
That's all these this is trying to do.

00:20:42.411 --> 00:20:47.061
So that AI method has to be better than
the covariate adjustments in our model,

00:20:48.351 --> 00:20:52.581
and I doubt it in most circumstances.

00:20:52.581 --> 00:20:57.561
I highly doubt in most scenarios that
we have that there might be some.

00:20:57.561 --> 00:20:59.791
And so we'll talk a little bit
about when that might be of value.

00:21:01.086 --> 00:21:01.806
So two things.

00:21:01.806 --> 00:21:06.966
First of all, if I have the data
that this unlearned machine has,

00:21:07.806 --> 00:21:09.966
I, we, we, we can do just as well.

00:21:10.236 --> 00:21:12.726
You, you could do, you
could do just as well.

00:21:12.936 --> 00:21:16.056
Now, I'm not saying barrier, you could
do just as well to use that as covariate.

00:21:16.236 --> 00:21:18.831
Figure out the right way
to model a covariate.

00:21:19.806 --> 00:21:25.206
And, uh, you know, does this deep neural
network do adjustments or predictions

00:21:25.206 --> 00:21:30.216
of that 18 month value better than
linear covariates or, or might,

00:21:30.216 --> 00:21:31.836
might even be quadratic covariates.

00:21:31.986 --> 00:21:34.956
You could learn based on that
data the right way to do this.

00:21:35.256 --> 00:21:36.396
I, I don't think so.

00:21:36.396 --> 00:21:38.736
I, I, it's, I think it's a lot of Blu.

00:21:39.916 --> 00:21:42.106
And I, it, it's, sure.

00:21:42.106 --> 00:21:43.636
I think it's very reasonable if you do it.

00:21:43.651 --> 00:21:48.466
I, I, I've got no qualms to it, and I
think it's a very reasonable thing to do.

00:21:48.646 --> 00:21:52.036
I just have my doubts as to
whether this is actually better.

00:21:52.771 --> 00:21:54.076
I, I, I, I doubt it.

00:21:54.106 --> 00:21:55.396
What, where's this scenario?

00:21:55.396 --> 00:21:56.506
It might be better.

00:21:56.746 --> 00:21:59.836
Let, let's be reasonable and
think about where's this scenario

00:21:59.836 --> 00:22:01.006
where this might be better.

00:22:01.666 --> 00:22:06.046
Suppose they have a couple hundred
patients in a rare disease that

00:22:06.046 --> 00:22:08.836
you don't have in a scenario.

00:22:09.261 --> 00:22:13.341
And you are not sure how to do
covariate adjustment in it, and

00:22:13.341 --> 00:22:15.441
they can provide this prediction.

00:22:16.281 --> 00:22:20.751
I think that could be of value, where
you don't have the data, you don't know

00:22:20.751 --> 00:22:22.401
how to do this covariate adjustment.

00:22:22.551 --> 00:22:27.081
This could be a very reasonable
thing to do, uh, in that scenario.

00:22:27.651 --> 00:22:32.511
Now, in a scenario, let's, let's think
of A-L-S-A-L-S, by the way, has this

00:22:32.511 --> 00:22:37.641
incredible resource, the ProAct database.

00:22:38.076 --> 00:22:43.191
Is this incredible resource in a
LS that has natural history of pat

00:22:43.386 --> 00:22:48.366
of, of a LS patients' behavior,
randomized placebos in there.

00:22:48.576 --> 00:22:51.666
They might even have active
treatments in this, this data set.

00:22:51.666 --> 00:22:55.716
I, I, I'm not sure of that, but
it's an incredible resource.

00:22:55.806 --> 00:23:00.456
I'm sure when Unlearn does
their, their, their deep neural

00:23:00.456 --> 00:23:03.156
networks, it's using that database.

00:23:04.401 --> 00:23:07.071
So I, I don't think there's value there.

00:23:07.071 --> 00:23:10.371
If you have the same data they
have in the scenario for a couple

00:23:10.371 --> 00:23:14.151
reasons, I think you could figure
out the right covariate adjustment

00:23:14.421 --> 00:23:16.671
times in since onset of disease.

00:23:16.671 --> 00:23:18.561
And a LS is a good predictor.

00:23:18.741 --> 00:23:22.391
Their their current A-L-S-F-R-S-I,
I I think you could do

00:23:22.391 --> 00:23:23.571
really, really well in there.

00:23:23.571 --> 00:23:25.971
And you don't need this AI powered thing.

00:23:25.971 --> 00:23:29.451
I'm not even sure it's better, but
they don't tell you what the model is.

00:23:29.991 --> 00:23:30.831
It's proprietary.

00:23:31.491 --> 00:23:34.581
So they give you a value of 3.1,

00:23:34.761 --> 00:23:36.861
but you don't know what the model is.

00:23:37.866 --> 00:23:42.486
I, I don't think that's the way forward
here, and I think it's a bad way

00:23:42.486 --> 00:23:46.596
forward with ai, um, in clinical trials.

00:23:46.596 --> 00:23:49.176
I think we need that model written down.

00:23:49.386 --> 00:23:55.206
I think we need that model to be
verified and, uh, uh, reprogrammed

00:23:55.206 --> 00:23:58.686
and, and, and, oh yeah, we generate,
we get the same value to that.

00:23:58.866 --> 00:24:02.166
I think if digital twins
go this route, that.

00:24:02.456 --> 00:24:06.656
It's this super secret deep neural
network thing, and it gives you a

00:24:06.656 --> 00:24:11.396
number that's bad and that's a bad
way forward for clinical trials.

00:24:11.396 --> 00:24:13.736
Now, I, I don't wanna fault unlearn.

00:24:13.736 --> 00:24:17.426
I, I, I think there's actually,
they may be doing things that are

00:24:17.426 --> 00:24:21.686
hugely valuable and even if that's
out there, people are gonna want

00:24:21.686 --> 00:24:24.206
to see it and use it in scenarios.

00:24:25.166 --> 00:24:29.606
So what are the benefits of
it is a covert adjustment.

00:24:29.906 --> 00:24:30.986
I think it's a lot of.

00:24:31.416 --> 00:24:35.916
Not, not much, uh, in there,
in, in these scenarios.

00:24:36.246 --> 00:24:39.606
Uh, so, but what, let's
think about the future.

00:24:39.606 --> 00:24:41.736
Let's think about where this is going.

00:24:43.506 --> 00:24:49.176
We are going to get really good at the
prediction of an individual patient level.

00:24:50.016 --> 00:24:53.706
I mentioned this ProAct database, a LS.

00:24:54.711 --> 00:24:59.751
In a clinical trial, when a patient
comes in, we can make really, really

00:24:59.751 --> 00:25:03.831
good prediction of where that patient
would be under natural history.

00:25:04.311 --> 00:25:05.481
Incredibly good.

00:25:06.456 --> 00:25:12.156
We don't actually take advantage of
that much in, uh, clinical trials.

00:25:12.186 --> 00:25:13.716
We're still randomizing.

00:25:13.716 --> 00:25:18.006
I'm not saying that's the wrong answer,
but we're gonna get really, really good.

00:25:18.006 --> 00:25:21.786
And I made, I, I was playing light
of it by saying, Hey Alexa, what

00:25:21.786 --> 00:25:23.346
would happen to this patient?

00:25:23.886 --> 00:25:26.046
We're gonna get really,
really good at that.

00:25:26.736 --> 00:25:29.856
And moving forward, we
have to utilize that.

00:25:30.276 --> 00:25:36.636
And we're gonna get really good at that
in a lot of different syndromes where

00:25:36.696 --> 00:25:42.426
we get a snapshot of a patient at a
point in time and we're gonna be able to

00:25:42.426 --> 00:25:44.226
have a really good prediction of that.

00:25:45.006 --> 00:25:46.896
But let's think about that a little bit.

00:25:46.896 --> 00:25:51.906
So, so going forward, I think this
whole digital twin technology, and we're

00:25:51.906 --> 00:25:55.836
gonna start calling it different things
because this can be slightly confusing,

00:25:56.316 --> 00:25:59.076
model based estimate of a patient.

00:26:00.096 --> 00:26:03.486
Are going to become incredibly
powerful things moving forward.

00:26:03.486 --> 00:26:08.166
It has to, now, we have to come to
grips with the causal aspects of this.

00:26:08.166 --> 00:26:14.496
We have to come to grips with the, the
bias in who enters a clinical trial,

00:26:14.496 --> 00:26:16.806
is it characterized by baseline?

00:26:17.256 --> 00:26:20.796
And there's still gonna be huge
value in randomization, for example.

00:26:20.796 --> 00:26:25.206
So imagine a clinical trial where you
do this and you randomize three to one.

00:26:25.641 --> 00:26:29.961
Treatment to control, and you've got
the model based predictions that are

00:26:29.961 --> 00:26:35.751
part of the primary analysis, but
you have real placebo patients to

00:26:35.751 --> 00:26:38.481
identify, are we doing this well?

00:26:39.171 --> 00:26:40.851
And they're randomized placebos.

00:26:40.851 --> 00:26:43.881
That's a patient that you enroll and
you say, we have a prediction of that

00:26:43.881 --> 00:26:47.391
patient if we do nothing to them,
and then we get the value of doing

00:26:47.391 --> 00:26:51.646
nothing to them, which can verify
and validate within your experiment.

00:26:52.296 --> 00:26:56.616
The whole notion of digital twins,
this becomes really, really powerful.

00:26:56.916 --> 00:27:03.036
So this, this combination of treatment
controls continually verifying and

00:27:03.036 --> 00:27:09.996
validating using less patients and,
and, and, and allowing a great deal of

00:27:09.996 --> 00:27:14.106
more certainty to that, to understand
whether a treatment is working.

00:27:14.706 --> 00:27:15.516
Is the future?

00:27:16.266 --> 00:27:18.996
If it's not the future, we're
not doing this right now.

00:27:19.116 --> 00:27:20.226
Where are we now?

00:27:20.256 --> 00:27:21.966
This is actually being done.

00:27:22.476 --> 00:27:26.406
It's not necessarily called this,
and that might be a good thing.

00:27:27.066 --> 00:27:31.026
So in a number of scenarios, we do this.

00:27:31.056 --> 00:27:33.781
Actually, propensity modeling is that.

00:27:35.316 --> 00:27:40.926
Where you have a historical database
and you wait certain patients to be

00:27:40.926 --> 00:27:47.016
a digital version of this patient,
we've just enrolled and then we wait.

00:27:47.016 --> 00:27:50.436
Their outcomes, it's digital twins.

00:27:51.726 --> 00:27:58.626
We do quite a bit at Barry, where we
model historical databases and we create

00:27:58.866 --> 00:28:01.266
a model based estimate of a patient.

00:28:01.266 --> 00:28:02.316
We do this in a LS.

00:28:03.546 --> 00:28:07.656
Where we create forecasts of a
patient enrolled in a trial and

00:28:07.656 --> 00:28:08.976
what they're going to look like.

00:28:09.336 --> 00:28:12.666
We do this in FTD, we
do this in Alzheimer's.

00:28:12.966 --> 00:28:16.176
We do this in a, in many diseases.

00:28:16.356 --> 00:28:18.396
We've done it 50 times.

00:28:19.596 --> 00:28:23.586
Now what's what's interesting
here is what do you do with this?

00:28:23.796 --> 00:28:29.976
I want you to think about a scenario where
what we get in a trial is a distribution.

00:28:31.881 --> 00:28:35.391
O and now I, I'm, I'm sure unlearn
has some level of prediction,

00:28:35.391 --> 00:28:36.891
variability and all of that.

00:28:36.891 --> 00:28:39.261
I don't think it's part
of the pro cova stuff.

00:28:39.351 --> 00:28:44.961
Uh, they, they have to do this to
some level, but it's not going to be

00:28:45.561 --> 00:28:48.411
that we simulate a single patient.

00:28:48.411 --> 00:28:52.011
As a payer, it's wholly inadequate.

00:28:53.751 --> 00:28:56.871
We need to simulate a million.

00:28:58.191 --> 00:29:00.681
And that's actually a distribution
that, that's what we know

00:29:00.681 --> 00:29:02.421
analytically as a distribution.

00:29:03.111 --> 00:29:08.961
So coming back to this issue of clinical
trial simulation where we do a lot

00:29:08.961 --> 00:29:12.921
of clinical trial simulation and we
have critics, I think Steven Sen has

00:29:12.921 --> 00:29:18.381
been a critic publicly on Twitter
of simulation, um, uh, within that.

00:29:18.381 --> 00:29:21.861
And he fits into this category of
this brilliant person that can do

00:29:21.861 --> 00:29:24.086
analytical things, uh, in that.

00:29:25.191 --> 00:29:32.601
We might be critics of, uh, a single
simulation of a single patient of,

00:29:33.021 --> 00:29:38.211
I want a million, I want a Google
number of them, which is actually

00:29:38.211 --> 00:29:42.651
just an analytical distribution for
every patient that walks in the door,

00:29:43.311 --> 00:29:45.561
Jane Doe comes into a clinical trial.

00:29:46.401 --> 00:29:49.281
We want a distribution of that patient.

00:29:49.371 --> 00:29:55.911
And the title of this, this, uh, in the
interim is digital Googols um, which

00:29:55.911 --> 00:30:03.091
is 10 to the hundred, uh, twins digital
Googols of this is just a distribution.

00:30:03.111 --> 00:30:06.831
It's an analytical
characterization of that patient.

00:30:07.881 --> 00:30:10.371
Now how do we, how do
we do this in a trial?

00:30:10.641 --> 00:30:12.171
How do we do P values?

00:30:12.231 --> 00:30:14.841
How do we do Bayesian
posterior probabilities?

00:30:15.021 --> 00:30:21.411
Now, in many of our trials, that model
that we've created has a mechanistic

00:30:21.411 --> 00:30:23.871
effect, like a rate of decline.

00:30:24.816 --> 00:30:29.796
Or as a mechanistic behavior
that going into the trial, we can

00:30:29.796 --> 00:30:35.076
actually include that as what is
the rate of decline for patients

00:30:35.076 --> 00:30:37.566
on a treatment relative to control?

00:30:37.926 --> 00:30:42.996
And we're estimating a theta
in a, in an oncology trial.

00:30:43.236 --> 00:30:49.776
Imagine if we have the, the behavior,
uh, and, and the overall survival.

00:30:50.631 --> 00:30:54.261
Distribution of a patient that
comes in the door that has a certain

00:30:54.261 --> 00:31:02.751
characteristic with a certain, uh,
uh, uh, cancer, we could then create

00:31:02.751 --> 00:31:06.891
this, this, this characterization,
this, this model-based estimate

00:31:06.891 --> 00:31:08.661
of their time of overall survival.

00:31:08.721 --> 00:31:14.121
And we can fit a hazard ratio based on the
individual patients that come into that.

00:31:14.601 --> 00:31:17.646
So that's a little bit
of one step forward.

00:31:18.546 --> 00:31:24.156
Of using the model based digital
virtualization of that patient

00:31:24.156 --> 00:31:28.866
to characterize treatment effect,
which can be incredibly powerful,

00:31:28.866 --> 00:31:35.376
especially under neurological
diseases, progressive diseases.

00:31:35.676 --> 00:31:37.656
This can be incredibly powerful.

00:31:37.866 --> 00:31:41.526
I'll throw out one where this,
this has to be the future is

00:31:41.856 --> 00:31:43.116
Duchenne muscular dystrophy.

00:31:44.331 --> 00:31:48.171
We've had a number of successful,
a number of controversial things.

00:31:48.381 --> 00:31:50.001
We have great data sources.

00:31:50.001 --> 00:31:53.361
They're actually not as
accessible as ProAct.

00:31:53.601 --> 00:31:57.261
It's actually one of the issues,
but it's going to become accessible.

00:31:57.261 --> 00:31:59.571
We're going to have these
phenomenal data sources.

00:31:59.571 --> 00:32:05.751
We've just gotta get rid of the roadblocks
to that, and we're going to use these, and

00:32:05.751 --> 00:32:10.491
we're gonna have this incredible virtual
characterization of a patient we know.

00:32:10.866 --> 00:32:14.646
What's gonna happen to a 4-year-old
boy with Duchenne muscular

00:32:14.646 --> 00:32:17.046
dystrophy that walks 240 meters?

00:32:17.466 --> 00:32:20.076
We can, we can show virtually
what's gonna happen.

00:32:20.286 --> 00:32:22.566
Can we characterize that
and put that in there?

00:32:23.226 --> 00:32:25.086
So it's a little bit of a different trial.

00:32:25.086 --> 00:32:27.906
The way forward of this is not, yeah.

00:32:27.906 --> 00:32:29.946
So think about what we do now in trials.

00:32:29.946 --> 00:32:34.656
We have a, a treatment and a placebo,
and we run things like T tests.

00:32:34.926 --> 00:32:36.876
We run MMRM models.

00:32:37.236 --> 00:32:38.376
We're not gonna do that.

00:32:38.901 --> 00:32:44.901
With a single paired observation
of that, it it in a, in a trial.

00:32:44.901 --> 00:32:48.591
Imagine you run a, a randomized
trial where we have a Google

00:32:49.191 --> 00:32:50.871
number of patients on placebo.

00:32:50.901 --> 00:32:54.591
That's what we have, and we're
gonna have to figure out the

00:32:54.591 --> 00:32:56.061
statistical way to do that.

00:32:56.061 --> 00:32:56.631
And we've done that.

00:32:56.631 --> 00:33:00.651
You can do non-parametric tests
where the, the, the, the model

00:33:01.521 --> 00:33:03.981
simulates a million observations.

00:33:04.791 --> 00:33:07.311
Of that single patient under control.

00:33:07.311 --> 00:33:11.451
And then you get the one observation
only in the physical system

00:33:11.451 --> 00:33:12.981
of that patient on treatment.

00:33:13.311 --> 00:33:18.051
But we've got no uncertainty in
this, this, this, uh, distribution.

00:33:18.681 --> 00:33:20.436
So these, these are gonna be really cool.

00:33:21.321 --> 00:33:23.961
And really cool trials going forward.

00:33:24.171 --> 00:33:28.881
I think that model has to
be written down, verifiable.

00:33:29.241 --> 00:33:32.451
Uh, people can go back and make
it better as we go forward.

00:33:32.451 --> 00:33:35.541
As you get more and more data,
it has to be included into this.

00:33:35.751 --> 00:33:37.941
Imagine this system what it is.

00:33:37.941 --> 00:33:40.671
It really is a disease.

00:33:40.671 --> 00:33:41.301
Alexa.

00:33:42.936 --> 00:33:45.546
What is going to happen
under this patient?

00:33:45.546 --> 00:33:50.496
And the characterization is
unbelievably promising within that.

00:33:50.526 --> 00:33:52.596
And I, I think that's the way forward.

00:33:52.926 --> 00:33:56.106
So in a way, I don't want you
to think, oh, Scott's, Scott's

00:33:56.106 --> 00:33:57.666
critical of digital twins.

00:33:57.666 --> 00:34:02.556
I, I might be the most excited person
out there for, for this virtualization

00:34:02.556 --> 00:34:07.596
of a patient in ways that I think
models can do this really, really well.

00:34:07.806 --> 00:34:09.726
And what it prevents is.

00:34:10.146 --> 00:34:15.306
Putting a lot of patients on a control,
think about how many patients we put on

00:34:15.306 --> 00:34:20.496
a control, and first of all, randomized
trials are so incredibly valuable.

00:34:20.496 --> 00:34:26.856
We're seeing the reason for it within
current healthcare policy and all of this.

00:34:26.856 --> 00:34:31.236
A complete lack of understanding
of the role of this, the need

00:34:31.236 --> 00:34:35.406
of it, the conclusions that
come out of it compared to.

00:34:36.561 --> 00:34:39.741
Misbehaving uses of, uh,
uh, other use of data.

00:34:39.741 --> 00:34:45.531
So randomized trials are, are absolutely
unbelievable and they won't go away.

00:34:46.161 --> 00:34:50.751
I think we'll continually randomize,
but this idea that five trials out there

00:34:50.991 --> 00:34:56.271
all randomized, 50% of their patients
and we're enrolling patients under a

00:34:56.271 --> 00:35:00.621
something and we know what's gonna happen
to them, we have an incredibly good idea.

00:35:00.771 --> 00:35:01.311
It's bad.

00:35:01.611 --> 00:35:05.751
It's incredibly inefficient as a
system, so we have to get there.

00:35:05.901 --> 00:35:07.641
We have to move there faster.

00:35:08.091 --> 00:35:12.646
Now it's only as good as the data and.

00:35:13.566 --> 00:35:15.996
20 years ago, it just didn't exist.

00:35:15.996 --> 00:35:18.636
We didn't really didn't
have these data sources.

00:35:18.906 --> 00:35:19.446
It exists.

00:35:19.446 --> 00:35:23.796
Some data sources existed within
pharma, some existed within the

00:35:23.796 --> 00:35:26.256
NIH, some existed at other places.

00:35:26.406 --> 00:35:27.726
You couldn't get your hands on it.

00:35:27.726 --> 00:35:28.896
We still have this issue.

00:35:29.286 --> 00:35:32.796
We still don't have great access
to data, but it's getting better.

00:35:33.996 --> 00:35:40.536
Uh, medical systems,
data sources out there.

00:35:41.376 --> 00:35:46.776
I, I think there's gonna be an explosion
of data, and it's going to be to the point

00:35:46.776 --> 00:35:48.636
that we don't know what to do with it.

00:35:48.846 --> 00:35:50.946
Let me give you another
circumstances of it.

00:35:50.946 --> 00:35:52.746
So, I, I wear a whoop band.

00:35:53.076 --> 00:35:58.836
It's like a, it's like a, a,
a, a watch, uh, a wearable that

00:35:58.836 --> 00:36:00.486
tracks all kinds of things about.

00:36:01.566 --> 00:36:07.386
Um, in, in terms of my heart rate,
my blood pressure, uh, my strain on

00:36:07.386 --> 00:36:12.786
the day, how much I sleep, it's gonna
characterize things about my behavior,

00:36:12.906 --> 00:36:17.676
how fast I walk, how big my gait is, how
much I type, all kinds of things that

00:36:17.676 --> 00:36:22.536
are gonna characterize people in ways
that what are we gonna do with this data?

00:36:22.536 --> 00:36:24.336
It's phenomenally powerful.

00:36:25.236 --> 00:36:28.446
So we're gonna need really
good models to do this.

00:36:28.446 --> 00:36:30.756
It might be neural networks that do this.

00:36:31.386 --> 00:36:36.006
Now they need to be
accessible, uh, within that.

00:36:36.006 --> 00:36:39.516
And if this goes, you know, they
need to be open source to some

00:36:39.516 --> 00:36:42.996
extent if we're gonna use them in
these incredible, powerful ways.

00:36:43.416 --> 00:36:43.711
So, oh,

00:36:45.881 --> 00:36:46.781
virtualization.

00:36:47.106 --> 00:36:50.526
Whether behind me is the
Pacific Ocean, you may not know.

00:36:50.871 --> 00:36:56.211
Um, uh, in these trials, but
this whole idea of virtualization

00:36:56.481 --> 00:37:00.321
as, as counterfactual in
trials is incredibly powerful.

00:37:00.741 --> 00:37:03.141
Uh, I, I think that this is the future.

00:37:03.141 --> 00:37:06.471
We're not, we are there
and this is being used.

00:37:06.471 --> 00:37:10.131
External data's being used in trials
in really, really powerful ways.

00:37:10.281 --> 00:37:11.841
It's, by the way, it's always been used.

00:37:11.841 --> 00:37:16.221
We, we've, we've approved drugs
based on single arm trials.

00:37:17.001 --> 00:37:20.781
And it's been compared to what we
think historically, what would've

00:37:20.781 --> 00:37:21.981
happened to those patients?

00:37:22.281 --> 00:37:27.501
An ORR rate that just seems that it
could never have happened based on

00:37:27.501 --> 00:37:29.301
historical ways to treat patients.

00:37:29.751 --> 00:37:33.771
We're getting better and better and better
at it's coming into rare diseases, it's

00:37:33.771 --> 00:37:35.571
gonna be coming into common diseases.

00:37:35.901 --> 00:37:39.801
It, and this notion as a system
of enrolling all these patients

00:37:39.801 --> 00:37:42.261
on a control just seems, wow.

00:37:42.621 --> 00:37:44.271
Uh, incredibly inefficient.

00:37:46.281 --> 00:37:51.351
So thank you for joining
me here in the interim.

00:37:52.251 --> 00:37:58.731
Until next time, I am Scott Berry,
your host, or at least you think I am.