WEBVTT

NOTE
This file was generated by Descript 

00:00:00.000 --> 00:00:03.089
Judith: Welcome to Berry's In the
Interim podcast, where we explore the

00:00:03.089 --> 00:00:07.400
cutting edge of innovative clinical
trial design for the pharmaceutical and

00:00:07.400 --> 00:00:09.420
medical industries, and so much more.

00:00:10.250 --> 00:00:10.930
Let's dive in.

00:00:12.562 --> 00:00:15.922
Well, welcome back
everyone to In The Interim.

00:00:16.322 --> 00:00:17.962
I'm your host, Scott Berry.

00:00:18.722 --> 00:00:22.242
And today I'm going to
touch on multiple topics.

00:00:22.252 --> 00:00:28.372
Those of you who join into this, uh,
podcast periodically know we like to

00:00:28.372 --> 00:00:33.052
delve into the world of sports and how
that's related to science and clinical

00:00:33.052 --> 00:00:35.402
trials and statistics and learning.

00:00:35.822 --> 00:00:39.752
So we're gonna, we're gonna dive back
and forth from sports to clinical

00:00:39.752 --> 00:00:42.582
trials here, and, uh, with it.

00:00:42.622 --> 00:00:47.622
And, and part of, uh, the, the
wonderful thing about this podcast

00:00:47.632 --> 00:00:51.402
is we've been doing this podcast,
uh, weekly, only missed a couple

00:00:51.402 --> 00:00:53.192
weeks within a couple years now.

00:00:53.622 --> 00:00:59.542
And I always wonder, "Okay, what's-- what,
what, what topics are coming next, uh,

00:00:59.542 --> 00:01:01.992
in the world of science and statistics?"

00:01:02.452 --> 00:01:05.812
And, you know, the, the
world brings topics.

00:01:05.842 --> 00:01:07.702
Uh, i- it's kind of amazing.

00:01:07.702 --> 00:01:12.462
And so I was, I was very struck by
this particular story, and I thought

00:01:12.462 --> 00:01:18.702
it was related to who I am as a
scientist and a statistician, and,

00:01:18.712 --> 00:01:21.692
um, see if you find it compelling.

00:01:22.172 --> 00:01:26.252
And the, the title of this is
Fairness in Soccer and Science.

00:01:26.742 --> 00:01:27.952
So I am not…

00:01:27.992 --> 00:01:32.882
Uh, and I'll say soccer in a
sense, um, uh, within this.

00:01:32.892 --> 00:01:35.482
So, uh, so let me get
to the story of this.

00:01:35.912 --> 00:01:41.552
Uh, those of you who can sort of see
where, where I'm, I'm currently located,

00:01:41.552 --> 00:01:49.272
I, I spend summers in Minnesota, and
beautiful woods, uh, surrounding me.

00:01:49.272 --> 00:01:51.182
You can sort of see the trees there.

00:01:51.572 --> 00:01:52.822
Gorgeous nature.

00:01:52.992 --> 00:01:58.232
Um, part of some of this gorgeous
nature is, is, is a negative.

00:01:58.642 --> 00:02:02.822
Uh, flying pests, insects, ticks.

00:02:03.172 --> 00:02:08.472
Uh, uh, northern Minnesota is,
is famous for its mosquitoes.

00:02:08.902 --> 00:02:13.212
Uh, i- it's, uh, for, for those
of you who have not been here,

00:02:13.442 --> 00:02:18.232
um, in the, in the summertime,
it can be, it can be really bad.

00:02:18.302 --> 00:02:22.422
At dusk, uh, in the evening,
it's, it's unbearable actually.

00:02:22.912 --> 00:02:24.252
Uh, horseflies.

00:02:25.002 --> 00:02:32.262
And so here we have in, in our cabin
here, uh, outside of our laundry

00:02:32.262 --> 00:02:40.342
room in the window, there is a
gorgeous, uh, spiderweb and, uh, a

00:02:40.642 --> 00:02:46.152
gorgeous spider that's right outside
the window and, and it lives there.

00:02:46.352 --> 00:02:48.122
And large spider.

00:02:48.172 --> 00:02:50.552
And by the way, they're,
they're incredible.

00:02:50.882 --> 00:02:52.702
Um, they're incredible.

00:02:52.702 --> 00:02:56.202
Um, um, I, I, I should be careful here.

00:02:56.202 --> 00:02:59.582
I, I, I don't think they're an
insect, but, but they're incredible

00:02:59.592 --> 00:03:01.702
beings and to watch them.

00:03:01.712 --> 00:03:03.402
So I enjoy watching them.

00:03:03.972 --> 00:03:13.114
And, um, I, I- I can help the spider by
turning on the light in the laundry room.

00:03:14.054 --> 00:03:18.964
So, uh, i- in the evening here, if
you flip on the light, of course, that

00:03:18.964 --> 00:03:25.914
attracts many insects to the window
and creates a, a huge benefit to that

00:03:25.914 --> 00:03:29.194
spider, uh, in terms of the flying things.

00:03:29.194 --> 00:03:34.604
I, I actually concern there are too
many, uh, at times, uh, given Minnesota.

00:03:35.024 --> 00:03:38.714
So is it cheating that,
that I turn the light on?

00:03:40.000 --> 00:03:44.490
And you know what, uh, now the
fact that I don't like the flying

00:03:44.490 --> 00:03:47.130
insects, um, within that, but in…

00:03:47.310 --> 00:03:51.720
A- and maybe that's me as a
statistician that, uh, I turn that

00:03:51.720 --> 00:03:53.790
on, and is that sort of cheating?

00:03:54.190 --> 00:03:59.660
Um, there's a spider elsewhere, um,
that doesn't have the light go on

00:03:59.660 --> 00:04:01.690
sort of thing in, in, in all of that.

00:04:01.690 --> 00:04:03.540
At least the thought crosses my mind.

00:04:04.040 --> 00:04:09.760
Yes, I, I do turn the light on, and
no, I don't have any issues with,

00:04:10.130 --> 00:04:17.130
with, um, um, helping that spider,
uh, do its job, uh, within it.

00:04:17.130 --> 00:04:19.340
I think it's a productive
thing around there.

00:04:19.340 --> 00:04:23.180
Now, it, it makes no difference
to us as humans to go outside.

00:04:23.180 --> 00:04:24.300
We don't notice anything.

00:04:24.300 --> 00:04:26.990
But, but I feel like spiders
are good things in that.

00:04:27.020 --> 00:04:33.630
But that, that thought goes through
my mind, um, uh, as to, I, I don't

00:04:33.630 --> 00:04:36.540
know, almost fairness, uh, within that.

00:04:37.200 --> 00:04:42.740
So what's the story that came up here,
uh, triggering the podcast topic?

00:04:43.750 --> 00:04:46.350
Um, I have been watching the World Cup.

00:04:46.390 --> 00:04:48.730
We are, we are in the
throes of the World Cup.

00:04:48.760 --> 00:04:54.680
I am not a, a, a football, I-
I'll say it that way, the, the

00:04:54.680 --> 00:04:56.310
way the rest of the world says it.

00:04:56.310 --> 00:04:57.860
The US calls it soccer.

00:04:58.250 --> 00:05:00.880
Uh, I'm not a big football fan.

00:05:00.890 --> 00:05:05.500
I, I enjoy watching it, but I
don't follow it tremendously.

00:05:05.760 --> 00:05:09.310
My kids played it, and I, I
kind of understand the rules.

00:05:09.470 --> 00:05:11.750
As statisticians, we, we love rules.

00:05:11.750 --> 00:05:14.410
A- and I love rules, and I understand it.

00:05:14.410 --> 00:05:16.270
I'm trying to understand
strategy and all that.

00:05:16.470 --> 00:05:22.550
And I was actually gonna do a podcast
on part of, uh, part of this that

00:05:22.600 --> 00:05:26.860
to a statistician and, and rules
that I don't like about soccer

00:05:26.860 --> 00:05:28.390
that I think could be improved.

00:05:28.660 --> 00:05:32.540
And now people, "Well, this is
tradition," and, you know, sort of thing.

00:05:32.810 --> 00:05:38.680
And an interesting thing is if,
if you get a penalty inside the,

00:05:38.690 --> 00:05:43.600
the large goal box, it's a penalty
kick, which is an enormous penalty.

00:05:43.920 --> 00:05:47.500
I don't know, 85% chance of a
goal or something like that.

00:05:48.080 --> 00:05:53.240
That can happen well away from the goal,
uh, right on the very corner of the

00:05:53.240 --> 00:05:58.290
box, and if you're one foot away from
that, it's a very different penalty.

00:05:58.290 --> 00:06:00.140
It's a much less severe penalty.

00:06:00.580 --> 00:06:05.340
There's dichotomization in soccer
that I think is, is not on the

00:06:05.370 --> 00:06:07.240
continuum, and I don't think is…

00:06:07.510 --> 00:06:08.740
I, I don't think it's a good rule.

00:06:08.760 --> 00:06:10.530
But that's not the topic of today.

00:06:11.000 --> 00:06:15.160
Sort of related to, to maybe what's
fair and all of that, and I, I think

00:06:15.160 --> 00:06:17.020
they don't call the same penalties.

00:06:17.340 --> 00:06:20.480
One foot difference, they would
call it a penalty, but not

00:06:20.480 --> 00:06:22.250
because of the severity of it.

00:06:22.520 --> 00:06:25.430
So it's, it's strange to
watch, to, to watch that.

00:06:26.250 --> 00:06:29.232
So I think that could be
improved But what is it?

00:06:29.242 --> 00:06:35.882
It, it is, it is the recent decision
of FIFA, the governing body of

00:06:35.892 --> 00:06:43.512
the World Cup, to suspend the red
card of Folarin Balogun, Balogun.

00:06:43.582 --> 00:06:45.882
I, I'm probably saying
his wr- his name wrong.

00:06:46.122 --> 00:06:53.452
He's, uh, the leading scorer for the
US, a striker for the US, and in the

00:06:53.482 --> 00:07:00.072
earlier game he got a, um, uh, a red card.

00:07:01.252 --> 00:07:05.912
During the game he, he had to
leave the game and they had to play

00:07:05.912 --> 00:07:08.462
actually one, one player short.

00:07:08.972 --> 00:07:12.452
And for about 25 minutes at the end,
they actually scored while they were

00:07:12.452 --> 00:07:16.742
playing short and, uh, they, they
gave up a goal but won the game.

00:07:17.112 --> 00:07:20.452
That player is then
suspended for the next game.

00:07:20.642 --> 00:07:24.872
Now the US gets to have their full
contingent of players, so they get to

00:07:24.872 --> 00:07:29.582
play all 11 players at this next one,
but, but the rules are if you get a red

00:07:29.582 --> 00:07:31.332
card you're suspended the next game.

00:07:32.892 --> 00:07:38.202
So FIFA announced that they're, they
are, uh, changing that and they're

00:07:38.202 --> 00:07:44.982
allowing him to play in the next game
against Belgium, and this, this, this

00:07:45.042 --> 00:07:50.072
podcast is, is going to come out after
that game, um, in terms of it, but

00:07:50.072 --> 00:07:51.922
it's being recorded before that game.

00:07:51.942 --> 00:07:54.872
It's actually the morning
of that game within it.

00:07:55.802 --> 00:08:01.792
Um, and it was announced the day before
that he was no longer suspended, and

00:08:01.792 --> 00:08:09.942
it was, it was reported that there was
contact with, um, United States President

00:08:09.952 --> 00:08:17.012
Trump asking them to look into this and
switched it, and the FIFA changed this.

00:08:17.182 --> 00:08:20.982
It's the first time in over 50
years that this has ever happened.

00:08:20.982 --> 00:08:22.672
It's never happened in the World Cup.

00:08:24.302 --> 00:08:26.782
So what, what…

00:08:27.412 --> 00:08:31.632
Yeah, it feels corrupt, and it
feels corrupt from the standpoint

00:08:31.642 --> 00:08:36.342
that had that been a player on
Belgium that would have happened,

00:08:37.072 --> 00:08:38.982
this would not have been changed.

00:08:39.352 --> 00:08:46.232
That player would not, um, be, be,
the, the resi- They suspended the

00:08:46.232 --> 00:08:49.872
decision, which is awkward because
they suspended the suspension, but,

00:08:50.282 --> 00:08:52.052
uh, is, is the wording they gave.

00:08:52.492 --> 00:08:54.182
That feels like that
wouldn't have happened.

00:08:54.252 --> 00:08:58.072
It feels like this is related
to the relationship of

00:08:58.092 --> 00:08:59.792
Trump to the FIFA president.

00:09:00.282 --> 00:09:02.502
The, the, the, the…

00:09:02.732 --> 00:09:03.572
This is

00:09:05.880 --> 00:09:08.070
influence, uh, from that.

00:09:08.120 --> 00:09:15.600
Now, I, I understand the red card call
was controversial within that, but it

00:09:16.160 --> 00:09:20.790
was the call and it was what happened,
and it was a video review that created

00:09:20.790 --> 00:09:24.660
that, uh, in it, and I think people
would argue, and I think generally people

00:09:24.660 --> 00:09:27.280
think it was somewhat of a bad call.

00:09:28.330 --> 00:09:33.210
But it was the call, and
to remove it feels unfair.

00:09:33.960 --> 00:09:35.530
Yeah, a little bit corrupt.

00:09:35.530 --> 00:09:38.290
It feels that this is not right.

00:09:39.130 --> 00:09:43.410
And it's interesting, much of the-
many of the reactions are almost

00:09:43.410 --> 00:09:45.200
feeling sorry for the US now.

00:09:45.220 --> 00:09:47.420
They, they, they're in a no-win situation.

00:09:47.810 --> 00:09:53.050
Should he score a goal in this
game, um, you know, it, it- we will

00:09:53.050 --> 00:09:57.880
always, the, the, the US soccer
team will always be felt like

00:10:00.056 --> 00:10:01.976
They had an unfair advantage.

00:10:02.386 --> 00:10:07.926
Uh, you can call it cheating, call it
what you will, that, that, that scarlet

00:10:07.926 --> 00:10:09.906
letter will be there no matter what.

00:10:10.476 --> 00:10:16.046
Um, it, it, it bothers me in my,
my watching of sports, and this is

00:10:16.046 --> 00:10:22.686
where this podcast goes, is this
issue of fairness and, and me as a, a

00:10:22.686 --> 00:10:25.016
statistician and, and what it means.

00:10:25.026 --> 00:10:29.026
So for example, I won't
be watching the game.

00:10:29.036 --> 00:10:36.786
There, there's something about that is,
is just so, so negative to me that this

00:10:36.796 --> 00:10:42.976
happens, that it feels like this is
now not a fair competition within it.

00:10:42.996 --> 00:10:47.966
And I, I don't wanna root against
the US, uh, in it, uh, but it…

00:10:47.966 --> 00:10:51.456
there's just so much of it that
I'd, I'd rather not watch it.

00:10:51.846 --> 00:10:55.266
I don't watch WWE.

00:10:55.616 --> 00:10:57.716
Um, this is wrestling.

00:10:58.136 --> 00:10:59.156
Um, all…

00:10:59.196 --> 00:10:59.806
I, you know, I…

00:10:59.876 --> 00:11:01.576
as a kid, I called it all-star wrestling.

00:11:01.666 --> 00:11:03.306
I, I don't even know where, where this is.

00:11:03.316 --> 00:11:07.206
It's scripted, uh, wrestling,
and it's entertainment, and I…

00:11:07.396 --> 00:11:08.756
you know, people can enjoy that.

00:11:08.756 --> 00:11:09.616
It's fantastic.

00:11:09.616 --> 00:11:11.756
They're phenomenal athletes, by the way.

00:11:12.176 --> 00:11:16.516
Um, stunning athletes,
but it is a performance.

00:11:16.546 --> 00:11:17.676
It's, it's Broadway.

00:11:17.686 --> 00:11:19.016
It's, it's scripted.

00:11:19.016 --> 00:11:21.626
I d- I, I just don't enjoy that.

00:11:21.626 --> 00:11:24.336
I, I don't care who wins, you
know, and all that, and that's

00:11:24.346 --> 00:11:25.486
not really the point of it.

00:11:25.526 --> 00:11:27.306
It's, it's a scripted performance.

00:11:27.306 --> 00:11:31.206
It's not, it's not sort of sport,
and at the center of sport has

00:11:31.206 --> 00:11:33.396
to be this level of fairness.

00:11:33.696 --> 00:11:37.176
And it's interesting because there was
an earlier game that I was watching, and

00:11:37.176 --> 00:11:39.186
I, I've been watching quite a bit of it.

00:11:39.246 --> 00:11:43.216
I, uh, it's been really,
really entertaining.

00:11:43.216 --> 00:11:45.046
It's been incredibly good sport.

00:11:45.046 --> 00:11:48.866
I, uh, I've, I've gra- gained
an appreciation for soccer.

00:11:49.006 --> 00:11:52.936
I'm not gonna call it football because
that puts me at some level that I,

00:11:52.936 --> 00:11:58.156
I, I, I'm now a, a, a knowledgeable
person on that, and I, I'm, I'm not.

00:11:58.156 --> 00:12:00.696
I don't, I don't wanna
be a pretender in that.

00:12:00.696 --> 00:12:03.326
But I've r- I've thoroughly
enjoyed it to this point.

00:12:03.576 --> 00:12:08.756
There was a game I was watching and,
and England was playing Croatia, and

00:12:09.006 --> 00:12:16.936
their superstar, uh, Harry Kane, had
a penalty kick and, um, missed it.

00:12:17.436 --> 00:12:19.236
Uh, the goalie saved it.

00:12:19.796 --> 00:12:26.786
They did a video review of it, and the
goalie left his spot too early, and that

00:12:27.586 --> 00:12:33.486
violation, and so on video review, they
gave him a second kick and he made it.

00:12:34.196 --> 00:12:40.818
And so he s- he scored And to
me, that was completely fair.

00:12:40.948 --> 00:12:46.088
There's a rule, and, uh, as long as
the rule is reinforced, and if it's

00:12:46.088 --> 00:12:49.008
reinforced by video, it, it got it right.

00:12:49.398 --> 00:12:54.008
And, you know, you could, you could
argue that that seems, that seems

00:12:54.268 --> 00:12:56.728
detailed or something, but it's the rule.

00:12:56.728 --> 00:12:57.468
And I, I…

00:12:57.768 --> 00:13:02.608
And I think that's related to who
I am as a statistician, that there

00:13:02.608 --> 00:13:07.078
are rules in the, uh, with- within
that, and that's completely fine.

00:13:07.458 --> 00:13:10.728
Um, and I thought I had
no problem with that.

00:13:10.728 --> 00:13:12.948
I don't feel that that's unfair.

00:13:13.178 --> 00:13:15.788
There's a set of rules,
and you follow the rules.

00:13:16.098 --> 00:13:21.468
And that's why this bothers me, that
he's, the, the striker is not suspended

00:13:21.468 --> 00:13:25.228
because there are rules and you follow
the rules, and we are not following

00:13:25.228 --> 00:13:29.024
the rules This is related to, to golf.

00:13:29.024 --> 00:13:34.534
If you watch a golf tournament, you'll
see many times a golfer is in a, a, a

00:13:34.534 --> 00:13:40.024
sort of bad position, but they're standing
on a sprinkler head, or their ball is

00:13:40.024 --> 00:13:45.874
in a particular position, there's a
camera in the way, and they are legally

00:13:45.874 --> 00:13:50.884
allowed to move their ball, and they
generally end up in a better position.

00:13:51.254 --> 00:13:55.424
It's not cheating, it is the rules, and
it's, it's important for every golfer

00:13:55.424 --> 00:13:59.484
to completely understand the rules,
and there are officials there that

00:13:59.484 --> 00:14:01.454
make sure everybody follows the rules.

00:14:01.714 --> 00:14:09.444
Golf is incredibly, um, um,
uh, rule-conscious in fairness.

00:14:09.634 --> 00:14:13.814
Cheating in golf, you're
essentially done as a golfer.

00:14:13.824 --> 00:14:17.754
I mean, you can't cheat in golf, and
they go out of their way to make sure

00:14:17.794 --> 00:14:23.204
to protect the field, and everybody,
your playing, uh, partner should be

00:14:23.204 --> 00:14:27.824
reinforcing the rules, and it's, it's
a really important part of golf is,

00:14:27.834 --> 00:14:32.654
is sort of fairness, so that Scottie
Scheffler, the number one ranked

00:14:32.654 --> 00:14:39.284
player in the world, doesn't get an
advantage over the 500th best player.

00:14:39.374 --> 00:14:41.374
They play under the same rules.

00:14:41.614 --> 00:14:42.604
I'll watch that.

00:14:42.684 --> 00:14:43.654
I enjoy that.

00:14:43.664 --> 00:14:46.694
That's sport, and, and that's fantastic.

00:14:47.024 --> 00:14:50.944
Um, American baseball now
has video review of pitches.

00:14:51.534 --> 00:14:56.354
There's a strike zone, and you
can, you can challenge that.

00:14:56.884 --> 00:14:58.804
I, I think it's great.

00:14:59.324 --> 00:15:02.164
Getting it right, having, having
the rules, I think it's actually

00:15:02.164 --> 00:15:06.054
made the umpires, who are, who are
fantastic at what they do, and they

00:15:06.064 --> 00:15:10.684
generally get it right, um, has
made them, have made them better.

00:15:10.904 --> 00:15:15.564
But all of this has, uh, has been a
very positive thing because everybody

00:15:15.564 --> 00:15:18.914
plays under the same rules within that.

00:15:18.924 --> 00:15:22.424
There's a fairness to it
which is critical in sports.

00:15:23.404 --> 00:15:23.824
Okay.

00:15:24.274 --> 00:15:27.224
So what, what, what does that mean?

00:15:27.224 --> 00:15:32.314
I've gone on for 15 minutes about
World Cup soccer and my reaction to it.

00:15:32.674 --> 00:15:39.084
Um, I, I, I think as a statistician now,
I don't know whether it's that I became

00:15:39.084 --> 00:15:45.634
a statistician because, uh, I, I saw
the rules and, and, and, and part of

00:15:45.634 --> 00:15:48.144
it, I almost think it's, it's science.

00:15:48.694 --> 00:15:54.414
Or being a scientist and working in
clinical trials make me understand and

00:15:54.414 --> 00:16:02.136
appreciate the, the importance of it
to Sports and clinical tri-- uh, to,

00:16:02.166 --> 00:16:06.816
to clinical trials that then relate
back to sports, and I have such an

00:16:06.886 --> 00:16:14.096
incredibly negative reaction to, um,
removing that suspension, um, uh, in

00:16:14.096 --> 00:16:15.876
it, and the lack of fairness to it.

00:16:16.696 --> 00:16:22.576
We as statisticians in clinical trials,
fairness are everything in clinical

00:16:22.576 --> 00:16:26.746
trials, and it is, it is science.

00:16:26.956 --> 00:16:31.236
We are, we are learning
about particular therapies.

00:16:31.276 --> 00:16:33.536
Are they safe and effective?

00:16:34.016 --> 00:16:36.366
Is A better than B?

00:16:36.616 --> 00:16:40.856
There's a lot of parallels between
sports and clinical trials.

00:16:41.816 --> 00:16:47.526
If there is undue cheating or
unfairness in a clinical trial,

00:16:48.426 --> 00:16:50.506
it means we don't know the answer.

00:16:50.506 --> 00:16:51.716
Is A better than B?

00:16:51.726 --> 00:16:56.336
Is, is, is, is this drug
safe and effective for a

00:16:56.336 --> 00:16:58.396
regulator to make a decision?

00:16:58.706 --> 00:17:03.856
Um, and let's leave out of it perhaps
influence in, in those decisions, which we

00:17:03.856 --> 00:17:10.346
all as clinical trialists, uh, you know,
abhor, uh, within that, um, uh, within it.

00:17:10.346 --> 00:17:17.266
And so there's such a parallel in what
we do in clinical trials to this question

00:17:17.266 --> 00:17:24.136
of fairness, and part of why I love it is
you can't fake science, uh, within that.

00:17:24.386 --> 00:17:30.366
Let's get out of the aspect that you
can fake data, and there's a great deal

00:17:30.366 --> 00:17:36.086
of, of things done in clinical trials
to make sure the data is honest data.

00:17:36.266 --> 00:17:38.046
It's reported as collected.

00:17:38.046 --> 00:17:41.436
So you could fake it by
presenting data that's not right.

00:17:41.636 --> 00:17:43.796
Everybody on the treatment lived.

00:17:43.966 --> 00:17:46.286
Everybody on the placebo died.

00:17:46.816 --> 00:17:48.456
The drug's a, a miracle.

00:17:48.696 --> 00:17:51.276
But, but short of that, you can't fake it.

00:17:51.626 --> 00:17:56.936
The data and the results of it,
the, the, the, the statistical

00:17:56.936 --> 00:18:02.526
analysis tied to it, demonstration,
demonstrating the likelihood of benefit

00:18:02.526 --> 00:18:04.956
of that treatment, uh, within it.

00:18:05.326 --> 00:18:10.606
There's this, uh, there, there's this
incredible structure and importance to

00:18:10.866 --> 00:18:16.916
things we do in clinical trials to create
this fairness, to answer these questions.

00:18:17.236 --> 00:18:24.696
Maybe that's why this sort of reacted--
I reacted so negatively, um, to that.

00:18:25.196 --> 00:18:32.246
It's, it's the, the, the same
reaction that I had to the whole

00:18:32.296 --> 00:18:37.396
aspect of the acetaminophen
and being the cause of autism.

00:18:38.396 --> 00:18:41.216
It's because it was not right.

00:18:41.586 --> 00:18:42.456
I'm just, just…

00:18:42.836 --> 00:18:48.236
Blatantly, it was a violation of
everything about science that we do.

00:18:48.866 --> 00:18:56.262
It was wanting it to be right
rather than it being right within

00:18:56.262 --> 00:19:01.982
the rules of science inferences
that we made, uh, uh, within it.

00:19:02.062 --> 00:19:05.512
It was not supported
by the data within it.

00:19:05.742 --> 00:19:11.332
Uh, vaccines causing autism
are not supported by the data.

00:19:11.722 --> 00:19:13.562
There's a want to it.

00:19:14.152 --> 00:19:19.602
It is-- It's, it's corruption within
it, and there's such a reaction.

00:19:19.602 --> 00:19:26.152
We should all react to that as
clinical trialists because it lowers

00:19:26.192 --> 00:19:31.362
everything about other conclusions
that we draw within that setting.

00:19:31.992 --> 00:19:39.162
And there is this battle, uh, within
it, um, that, that we continually go on

00:19:39.162 --> 00:19:45.292
to, to be able to present this as fair,
to learn the truth about the therapies

00:19:45.342 --> 00:19:46.892
out there that we're looking at.

00:19:48.572 --> 00:19:49.002
Now

00:19:52.692 --> 00:19:58.592
We, we, the, the, this adjudication
of a clinical trial, um,

00:19:58.682 --> 00:20:01.142
within it is A better than B.

00:20:01.142 --> 00:20:06.482
We run these contests, and there is this
sort of analogy to me of sports, and

00:20:06.482 --> 00:20:09.762
maybe it's why I, I, I love what I do.

00:20:09.762 --> 00:20:10.762
I love sports.

00:20:10.762 --> 00:20:12.562
I love clinical trials.

00:20:13.062 --> 00:20:17.042
I, I love seeing the
results of these trials.

00:20:17.042 --> 00:20:20.232
I love seeing the
inferences on these trials.

00:20:20.672 --> 00:20:23.382
What, what conclusions are we making?

00:20:23.632 --> 00:20:28.942
Are we understanding sort of what, what's
fair and how to make these conclusions?

00:20:29.452 --> 00:20:34.522
A little bit I'll go back, when
I'm asked to present to non-science

00:20:34.562 --> 00:20:41.622
people, students, a- anybody about
statistics, I generally don't go

00:20:41.622 --> 00:20:44.672
in and present quantitative things.

00:20:44.912 --> 00:20:45.922
My draw…

00:20:46.152 --> 00:20:50.542
Two, two exam- two examples I go
back to that I think are just so,

00:20:51.172 --> 00:20:55.762
so awesome examples, and they're
very, very simple examples.

00:20:55.762 --> 00:21:02.492
One is Sherlock Holmes, and it's--
He, he, uh, he wr- Arthur Conan

00:21:02.492 --> 00:21:05.032
Doyle wrote about Sherlock Holmes.

00:21:05.032 --> 00:21:13.502
It's a sor- story, Silver Blaze, 1892
short story, where there's a disappearance

00:21:13.502 --> 00:21:16.612
of a valuable racehorse within it.

00:21:17.102 --> 00:21:23.922
And, um, the Scotland Yard detective
asks, "Is there any other point

00:21:23.942 --> 00:21:29.182
to wish to, to which you would
wish to draw my attention?"

00:21:30.532 --> 00:21:37.242
And Sherlock Holmes says, "To the curious
incident of the dog in the nighttime."

00:21:39.414 --> 00:21:45.854
And the Scotland Yard detective says,
"The dog did nothing in the nighttime."

00:21:47.074 --> 00:21:51.654
And Sherlock Holmes says, "That
was the curious incident."

00:21:53.124 --> 00:22:00.064
And it was the lack of barking
by a dog that's out there that

00:22:00.064 --> 00:22:03.324
would bark if somebody was there.

00:22:04.534 --> 00:22:09.864
That w- the lack of information was
the incredible information there,

00:22:10.464 --> 00:22:15.264
and that's something as statisticians
that it's so important in what we do.

00:22:15.264 --> 00:22:19.714
It's not just the data, it's
how did we get the data?

00:22:19.724 --> 00:22:22.034
What data did we not see?

00:22:22.234 --> 00:22:27.124
How, uh, you know, was
this prospectively defined?

00:22:27.604 --> 00:22:32.294
Was this one of 1,000 things
done and presented to us?

00:22:32.534 --> 00:22:34.534
That has very different meaning.

00:22:34.714 --> 00:22:39.154
The data are the same, very different
meaning to us making inferences

00:22:39.154 --> 00:22:44.074
about something than if it was
the primary analysis in the trial.

00:22:44.274 --> 00:22:49.434
These are so critically important
within it, but it can be

00:22:49.434 --> 00:22:51.724
presented differently to us.

00:22:52.274 --> 00:22:58.554
And so that example of making inferences
from data, I, I feel like is what is

00:22:58.554 --> 00:23:03.794
so attractive to me about science,
about what I do as a statistician.

00:23:04.724 --> 00:23:06.634
The other example is the…

00:23:06.664 --> 00:23:08.894
And this is also a famous example.

00:23:09.124 --> 00:23:12.504
Um, by the way, I don't know if this
example, you know what I mean, the whole

00:23:12.504 --> 00:23:17.404
sense of this is whether this is a, a tall
tale of, of it, but it's such a brilliant

00:23:17.404 --> 00:23:20.164
example, uh, I'll assume it- it's true.

00:23:20.564 --> 00:23:29.654
It's a story of Abraham Wald making,
um, looking to reinforce fighter j-

00:23:29.674 --> 00:23:36.024
fighter planes in World War II, and
the data being collected were the

00:23:36.024 --> 00:23:42.104
location of bullet holes in planes
when they went out and during a battle.

00:23:42.714 --> 00:23:49.014
Within it, you, they, the location, you,
they got data on where the holes were

00:23:49.054 --> 00:23:56.692
on the plane And the, the fascinating
thing was his conclusion was not to

00:23:56.692 --> 00:24:02.892
reinforce where the bullet holes were,
but to reinforce where they're not.

00:24:04.442 --> 00:24:05.402
Why is that?

00:24:05.402 --> 00:24:11.942
Because of the survivorship bias
that you only see the planes that

00:24:11.942 --> 00:24:17.852
survive and come back, and they have
bullet holes in certain locations.

00:24:18.182 --> 00:24:21.092
What you don't see are
the planes that go down.

00:24:21.092 --> 00:24:22.062
You'd love to see…

00:24:22.062 --> 00:24:26.272
The best data would be, let me
see where the location is of the

00:24:26.282 --> 00:24:28.422
bullet holes that took planes down.

00:24:28.722 --> 00:24:30.312
That's where you wanna reinforce.

00:24:30.452 --> 00:24:31.602
You don't see that data.

00:24:32.072 --> 00:24:34.472
You see planes that survive.

00:24:34.502 --> 00:24:36.442
There's a survivorship bias.

00:24:36.972 --> 00:24:41.422
And so it's almost as though where
those bullet holes are didn't take the

00:24:41.422 --> 00:24:46.762
plane down, but assuming a uniformity
of bullet holes over the plane, and

00:24:46.762 --> 00:24:51.232
there, there's an assumption there,
it's where they're not, uh, within it.

00:24:51.242 --> 00:24:58.332
Such a cool example of inference from data
and science, uh, wi- within that scenario.

00:24:58.862 --> 00:25:02.072
And I believe these are the kinds
of things in clinical trials.

00:25:02.072 --> 00:25:06.862
By the way, this survivorship shows up as
incredibly important thing in making me-

00:25:06.952 --> 00:25:10.122
in medical decision-making within that.

00:25:10.322 --> 00:25:13.512
And I think as scientists in
clinical trial, not, not…

00:25:13.542 --> 00:25:16.962
I, I mean, this is, this is also,
it's not just statisticians.

00:25:17.392 --> 00:25:18.042
We, we…

00:25:18.042 --> 00:25:21.412
This is something we very much deal
with, but clinical trial scientists,

00:25:21.422 --> 00:25:27.222
clinicians, everybody involved in
the enterprise of clinical trials and

00:25:27.222 --> 00:25:32.992
medical decision-making, th- this is
such an important part, uh, is making

00:25:33.002 --> 00:25:36.294
inferences from the data Within that.

00:25:36.294 --> 00:25:42.114
So that's-- It's the joy of,
of, for me, clinical trials and

00:25:42.114 --> 00:25:43.924
science and the importance of it.

00:25:44.434 --> 00:25:50.024
And there are a lot of parts of, of
clinical trials that, um, i-in some

00:25:50.024 --> 00:25:53.844
sense, people maybe think are un-
uh, uh, you know, not appropriate.

00:25:54.514 --> 00:26:01.774
Uh, many times, uh, running clinical
trials, sponsors will look at subsets,

00:26:01.794 --> 00:26:09.654
look at other endpoints, and see data that
looks really good and believe the result

00:26:09.654 --> 00:26:17.634
of that, not understanding the process
of looking at many things is a real, uh,

00:26:17.814 --> 00:26:20.344
a part about drawing those conclusions.

00:26:20.664 --> 00:26:25.834
You're very likely to find
something positive which lowers

00:26:25.844 --> 00:26:28.734
the inferential strength of that.

00:26:28.764 --> 00:26:33.954
Doesn't mean it's wrong, but it lowers the
inferential strength of that, uh, in that.

00:26:33.964 --> 00:26:39.494
So these post-hoc analyses, i-it's, it's
part of the science of these conclusion.

00:26:40.394 --> 00:26:44.194
Survivorship bias absolutely
shows up in clinical trials.

00:26:44.194 --> 00:26:50.084
If you're comparing to a natural
history s- database and you have a

00:26:50.114 --> 00:26:56.294
clinical trial where people enter
the clinical trial, there are people

00:26:56.294 --> 00:27:00.214
in the natural history study that
wouldn't get in the clinical trial.

00:27:00.474 --> 00:27:03.854
Maybe they die before
they would get the drug.

00:27:04.434 --> 00:27:10.974
And if somebody has the disease and has a
period of time before they are randomized

00:27:10.974 --> 00:27:19.794
in the trial, and now they take the drug,
it creates a survivorship bias for people

00:27:19.794 --> 00:27:25.464
that took the drug that somebody in the
natural history that had the disease that

00:27:25.474 --> 00:27:29.824
died before they'd get the drug would
be in the natural history study, but

00:27:29.824 --> 00:27:31.724
they wouldn't be in the clinical trial.

00:27:32.074 --> 00:27:39.704
And then the clinical trial, uh, uh,
people, uh, patients do better in

00:27:39.704 --> 00:27:42.434
comparison to the natural history study.

00:27:42.694 --> 00:27:45.544
Such a clear absolute survivorship bias.

00:27:45.544 --> 00:27:49.444
It shows up in many times about
making inferences about data.

00:27:49.824 --> 00:27:54.884
It's about fairness i-i-in, in
all of this, and it's-- We, we are

00:27:54.884 --> 00:27:57.464
the officers of, of this fairness.

00:27:58.044 --> 00:28:05.374
Now interestingly, uh, I, I and, and, uh,
Berry Consultants many times are involved

00:28:05.814 --> 00:28:09.724
in non-standard clinical trial stuff.

00:28:09.974 --> 00:28:14.804
We enjoy being involved
in hard clinical trials.

00:28:15.124 --> 00:28:21.974
In the most sort of standard clinical
trial that you enroll two arms to

00:28:21.974 --> 00:28:27.664
a fixed sample size, and you do
an analysis, no interim analysis.

00:28:27.674 --> 00:28:32.964
In some sense, the, the, the,
the cleanest data you can have.

00:28:33.764 --> 00:28:36.544
Um, and it's awesome.

00:28:36.864 --> 00:28:42.434
The, the, these trials are awesome,
but we're typically involved in

00:28:42.434 --> 00:28:46.304
adaptive designs, Bayesian analyses.

00:28:46.644 --> 00:28:52.174
The pod-- The last episode of, of
the-- In the Interim talked about

00:28:52.554 --> 00:28:57.314
bias in the estimate of the treatment
effect when you have futility stopping

00:28:57.334 --> 00:29:02.524
or superiority stopping and, and,
and the biased estimate of that.

00:29:02.524 --> 00:29:04.754
And by the way, it's, it's very different.

00:29:04.754 --> 00:29:06.134
It doesn't mean it's wrong.

00:29:06.524 --> 00:29:09.984
It has-- This has influence,
and when we get the result,

00:29:09.984 --> 00:29:14.904
we understand the influence of
that, uh, completely within that.

00:29:15.314 --> 00:29:17.094
Borrowing from subgroups.

00:29:17.104 --> 00:29:22.064
If you borrow from an adult
to a pediatric, uh, patient,

00:29:22.074 --> 00:29:23.974
it creates bias in that.

00:29:24.294 --> 00:29:30.074
Um, if you borrow from group A to group
B, you use controls from a different

00:29:30.554 --> 00:29:35.134
era, uh, in a platform trial within that.

00:29:35.164 --> 00:29:37.374
Is there potential bias in that?

00:29:37.804 --> 00:29:45.094
Now, all of these things, you know, maybe
it feels strange that I started this off

00:29:45.114 --> 00:29:53.204
with this very high level, um, that this
feels unfair that this player is allowed

00:29:53.244 --> 00:29:59.466
to play in the game W-within the setting,
but yet now talking about adaptive

00:29:59.466 --> 00:30:01.356
designs and Bayesian and all that.

00:30:02.286 --> 00:30:10.446
It, it's, it, it's incredibly,
um, important that we can't

00:30:10.456 --> 00:30:12.826
keep running those trials.

00:30:12.886 --> 00:30:14.696
We miss shots on goal.

00:30:14.856 --> 00:30:17.176
There are drugs that are not explored.

00:30:17.516 --> 00:30:21.656
We have to be efficient in
patient and time and resources.

00:30:22.326 --> 00:30:26.166
Uh, otherwise, we get three
really good answers, and we didn't

00:30:26.196 --> 00:30:28.756
ask 100 questions within that.

00:30:29.396 --> 00:30:30.906
There's an efficiency to this.

00:30:30.926 --> 00:30:34.646
Now, this can be, and it is science.

00:30:35.406 --> 00:30:36.676
Bayesian is science.

00:30:36.676 --> 00:30:38.826
Adaptive designs is science.

00:30:38.936 --> 00:30:43.336
Pre-specification in these settings
is so incredibly important.

00:30:43.826 --> 00:30:49.256
Covariate adjustment is science, and
it's incredibly important in that.

00:30:49.256 --> 00:30:53.606
It's not cheating to adjust for
covariates within the setting.

00:30:53.856 --> 00:30:58.136
You might look at the data as the
raw data of A and B, but a covariate

00:30:58.136 --> 00:31:00.106
adjustment gives you a different answer.

00:31:00.826 --> 00:31:01.796
Is that cheating?

00:31:01.806 --> 00:31:02.156
No.

00:31:02.336 --> 00:31:04.566
We understand that as science.

00:31:04.636 --> 00:31:07.026
We can have very clean science.

00:31:07.046 --> 00:31:08.506
Let me give you another example.

00:31:08.506 --> 00:31:13.716
Suppose you were collecting data, and
you were comparing to natural history.

00:31:15.366 --> 00:31:21.076
We understand, and I just presented how
there's a survivorship bias in comparing

00:31:21.086 --> 00:31:26.456
those natural history patients to
patients in a clinical trial setting.

00:31:28.276 --> 00:31:33.736
Does that mean we can't use it, or does
that mean we could potentially adjust

00:31:34.286 --> 00:31:37.236
and estimate the survivorship bias?

00:31:38.696 --> 00:31:46.026
That can be done in very clean scientific
ways that are honest and, and is

00:31:46.026 --> 00:31:53.116
science, and that's where w-we, we
kind of live on this, this border and,

00:31:53.126 --> 00:31:58.666
and, and the hard part is that's not
cheating, uh, uh, within the setting.

00:31:58.906 --> 00:32:05.896
And so, um, adjusting for it,
understanding the bias and early stopping

00:32:06.086 --> 00:32:11.776
and understand making conclusions from
that is still incredibly important, and

00:32:11.776 --> 00:32:18.666
it's an incredibly positive thing for
clinical trial science, and that's kind of

00:32:18.666 --> 00:32:23.826
the s- the space we live in, and I think
the importance of us sharing these ideas

00:32:23.826 --> 00:32:26.686
and presenting this, uh, uh, within it

00:32:29.312 --> 00:32:35.512
Now, it's, it's the evolution
of the standard trial that's 300

00:32:35.512 --> 00:32:40.392
patients, five years, look at the
data, make a conclusion within that.

00:32:40.822 --> 00:32:43.372
This can all be done and is fair.

00:32:43.552 --> 00:32:48.792
It's a little bit like the advances
of adding video review to it, of,

00:32:48.822 --> 00:32:51.602
of new rules within the setting.

00:32:51.802 --> 00:32:56.262
It's different than it used to be, but
it can all be done in a, in a fair way.

00:32:56.522 --> 00:33:03.192
There could be an avenue where a red
card is reversed as long as it goes

00:33:03.192 --> 00:33:05.292
in the rules of how this is done.

00:33:05.502 --> 00:33:10.832
And within this particular setting,
it, it went outside the boundaries.

00:33:10.842 --> 00:33:14.762
There isn't even really a
setting for that, uh, within it,

00:33:15.592 --> 00:33:18.502
which hence that feels unfair.

00:33:20.462 --> 00:33:24.952
So we all as statisticians,
clinicians, clinical trial scientists,

00:33:24.952 --> 00:33:31.592
regulators, sites, uh, all of this,
we're all officers of science, uh,

00:33:31.632 --> 00:33:33.762
in this, and science is evolving.

00:33:34.062 --> 00:33:39.152
We are making this machinery better and
better and better, and it, it improves

00:33:39.152 --> 00:33:44.072
the human condition to understand
the truth about different treatments,

00:33:44.102 --> 00:33:46.562
the, the efficacy and safety of it.

00:33:46.932 --> 00:33:52.012
It's, it's all such an important thing,
and maybe that's why I reacted to that

00:33:52.012 --> 00:33:54.322
particular case in, in such a way.

00:33:54.632 --> 00:34:00.792
Or maybe it's because, uh, I became a
statistician because I reacted to that.

00:34:00.882 --> 00:34:04.892
Uh, I, I, I don't know which
one of those it is, but I, I did

00:34:04.892 --> 00:34:06.632
have such a reaction to that.

00:34:08.012 --> 00:34:12.332
So everybody out there, keep up
the incredible important work

00:34:12.332 --> 00:34:15.942
you do, um, uh, within this.

00:34:16.332 --> 00:34:21.162
And until the next time, we
will be here in the interim