Fairness in Soccer and Clinical Trials

In this episode of "In the Interim...", Dr. Scott Berry investigates the practical meaning of fairness by connecting a controversial World Cup soccer ruling to foundational questions in clinical trial statistics. Scott scrutinizes FIFA’s unusual reversal of a red card suspension for US striker Folarin Balogun, referencing reports of US presidential influence, and draws explicit parallels between the enforcement of rules in international sport and the necessity for rigorously defined procedures in science. He references how systems thrive, or fail, on clear, consistently applied standards. Using Sherlock Holmes’ “Silver Blaze” and Abraham Wald’s WWII aircraft analysis, Scott revisits core statistical ideas about inference and missing data, survivorship bias, and the difference between prespecified versus post-hoc analyses. This episode affirms that adaptive and Bayesian approaches, when built on sound pre-specification and methodological discipline, represent scientific progress, offering a measured perspective on how standards and expectations of fairness continue to evolve.

Key Highlights:

• FIFA’s red card reversal, reports of external influence, and ramifications for procedural legitimacy
• Analogies from soccer, golf, baseball, and wrestling on the societal role of rules and enforcement
• Classic statistics lessons on missing data, inference, and survivorship bias
• Discussion of post-hoc versus prespecified analysis and its implications in trial integrity
• Defense of adaptive and Bayesian methodology as scientifically valid through pre-specification and covariate adjustment
• Reflection on the ongoing evolution of fairness and rigor in sport and science

For more, visit us at https://www.berryconsultants.com/

Creators and Guests

Host

Scott Berry

President and a Senior Statistical Scientist at Berry Consultants, LLC

What is In the Interim...?

A podcast on statistical science and clinical trials.

Explore the intricacies of Bayesian statistics and adaptive clinical trials. Uncover methods that push beyond conventional paradigms, ushering in data-driven insights that enhance trial outcomes while ensuring safety and efficacy. Join us as we dive into complex medical challenges and regulatory landscapes, offering innovative solutions tailored for pharma pioneers. Featuring expertise from industry leaders, each episode is crafted to provide clarity, foster debate, and challenge mainstream perspectives, ensuring you remain at the forefront of clinical trial excellence.

Judith: Welcome to Berry's In the
Interim podcast, where we explore the

cutting edge of innovative clinical
trial design for the pharmaceutical and

medical industries, and so much more.

Let's dive in.

Well, welcome back
everyone to In The Interim.

I'm your host, Scott Berry.

And today I'm going to
touch on multiple topics.

Those of you who join into this, uh,
podcast periodically know we like to

delve into the world of sports and how
that's related to science and clinical

trials and statistics and learning.

So we're gonna, we're gonna dive back
and forth from sports to clinical

trials here, and, uh, with it.

And, and part of, uh, the, the
wonderful thing about this podcast

is we've been doing this podcast,
uh, weekly, only missed a couple

weeks within a couple years now.

And I always wonder, "Okay, what's-- what,
what, what topics are coming next, uh,

in the world of science and statistics?"

And, you know, the, the
world brings topics.

Uh, i- it's kind of amazing.

And so I was, I was very struck by
this particular story, and I thought

it was related to who I am as a
scientist and a statistician, and,

um, see if you find it compelling.

And the, the title of this is
Fairness in Soccer and Science.

So I am not…

Uh, and I'll say soccer in a
sense, um, uh, within this.

So, uh, so let me get
to the story of this.

Uh, those of you who can sort of see
where, where I'm, I'm currently located,

I, I spend summers in Minnesota, and
beautiful woods, uh, surrounding me.

You can sort of see the trees there.

Gorgeous nature.

Um, part of some of this gorgeous
nature is, is, is a negative.

Uh, flying pests, insects, ticks.

Uh, uh, northern Minnesota is,
is famous for its mosquitoes.

Uh, i- it's, uh, for, for those
of you who have not been here,

um, in the, in the summertime,
it can be, it can be really bad.

At dusk, uh, in the evening,
it's, it's unbearable actually.

Uh, horseflies.

And so here we have in, in our cabin
here, uh, outside of our laundry

room in the window, there is a
gorgeous, uh, spiderweb and, uh, a

gorgeous spider that's right outside
the window and, and it lives there.

And large spider.

And by the way, they're,
they're incredible.

Um, they're incredible.

Um, um, I, I, I should be careful here.

I, I, I don't think they're an
insect, but, but they're incredible

beings and to watch them.

So I enjoy watching them.

And, um, I, I- I can help the spider by
turning on the light in the laundry room.

So, uh, i- in the evening here, if
you flip on the light, of course, that

attracts many insects to the window
and creates a, a huge benefit to that

spider, uh, in terms of the flying things.

I, I actually concern there are too
many, uh, at times, uh, given Minnesota.

So is it cheating that,
that I turn the light on?

And you know what, uh, now the
fact that I don't like the flying

insects, um, within that, but in…

A- and maybe that's me as a
statistician that, uh, I turn that

on, and is that sort of cheating?

Um, there's a spider elsewhere, um,
that doesn't have the light go on

sort of thing in, in, in all of that.

At least the thought crosses my mind.

Yes, I, I do turn the light on, and
no, I don't have any issues with,

with, um, um, helping that spider,
uh, do its job, uh, within it.

I think it's a productive
thing around there.

Now, it, it makes no difference
to us as humans to go outside.

We don't notice anything.

But, but I feel like spiders
are good things in that.

But that, that thought goes through
my mind, um, uh, as to, I, I don't

know, almost fairness, uh, within that.

So what's the story that came up here,
uh, triggering the podcast topic?

Um, I have been watching the World Cup.

We are, we are in the
throes of the World Cup.

I am not a, a, a football, I-
I'll say it that way, the, the

way the rest of the world says it.

The US calls it soccer.

Uh, I'm not a big football fan.

I, I enjoy watching it, but I
don't follow it tremendously.

My kids played it, and I, I
kind of understand the rules.

As statisticians, we, we love rules.

A- and I love rules, and I understand it.

I'm trying to understand
strategy and all that.

And I was actually gonna do a podcast
on part of, uh, part of this that

to a statistician and, and rules
that I don't like about soccer

that I think could be improved.

And now people, "Well, this is
tradition," and, you know, sort of thing.

And an interesting thing is if,
if you get a penalty inside the,

the large goal box, it's a penalty
kick, which is an enormous penalty.

I don't know, 85% chance of a
goal or something like that.

That can happen well away from the goal,
uh, right on the very corner of the

box, and if you're one foot away from
that, it's a very different penalty.

It's a much less severe penalty.

There's dichotomization in soccer
that I think is, is not on the

continuum, and I don't think is…

I, I don't think it's a good rule.

But that's not the topic of today.

Sort of related to, to maybe what's
fair and all of that, and I, I think

they don't call the same penalties.

One foot difference, they would
call it a penalty, but not

because of the severity of it.

So it's, it's strange to
watch, to, to watch that.

So I think that could be
improved But what is it?

It, it is, it is the recent decision
of FIFA, the governing body of

the World Cup, to suspend the red
card of Folarin Balogun, Balogun.

I, I'm probably saying
his wr- his name wrong.

He's, uh, the leading scorer for the
US, a striker for the US, and in the

earlier game he got a, um, uh, a red card.

During the game he, he had to
leave the game and they had to play

actually one, one player short.

And for about 25 minutes at the end,
they actually scored while they were

playing short and, uh, they, they
gave up a goal but won the game.

That player is then
suspended for the next game.

Now the US gets to have their full
contingent of players, so they get to

play all 11 players at this next one,
but, but the rules are if you get a red

card you're suspended the next game.

So FIFA announced that they're, they
are, uh, changing that and they're

allowing him to play in the next game
against Belgium, and this, this, this

podcast is, is going to come out after
that game, um, in terms of it, but

it's being recorded before that game.

It's actually the morning
of that game within it.

Um, and it was announced the day before
that he was no longer suspended, and

it was, it was reported that there was
contact with, um, United States President

Trump asking them to look into this and
switched it, and the FIFA changed this.

It's the first time in over 50
years that this has ever happened.

It's never happened in the World Cup.

So what, what…

Yeah, it feels corrupt, and it
feels corrupt from the standpoint

that had that been a player on
Belgium that would have happened,

this would not have been changed.

That player would not, um, be, be,
the, the resi- They suspended the

decision, which is awkward because
they suspended the suspension, but,

uh, is, is the wording they gave.

That feels like that
wouldn't have happened.

It feels like this is related
to the relationship of

Trump to the FIFA president.

The, the, the, the…

This is

influence, uh, from that.

Now, I, I understand the red card call
was controversial within that, but it

was the call and it was what happened,
and it was a video review that created

that, uh, in it, and I think people
would argue, and I think generally people

think it was somewhat of a bad call.

But it was the call, and
to remove it feels unfair.

Yeah, a little bit corrupt.

It feels that this is not right.

And it's interesting, much of the-
many of the reactions are almost

feeling sorry for the US now.

They, they, they're in a no-win situation.

Should he score a goal in this
game, um, you know, it, it- we will

always, the, the, the US soccer
team will always be felt like

They had an unfair advantage.

Uh, you can call it cheating, call it
what you will, that, that, that scarlet

letter will be there no matter what.

Um, it, it, it bothers me in my,
my watching of sports, and this is

where this podcast goes, is this
issue of fairness and, and me as a, a

statistician and, and what it means.

So for example, I won't
be watching the game.

There, there's something about that is,
is just so, so negative to me that this

happens, that it feels like this is
now not a fair competition within it.

And I, I don't wanna root against
the US, uh, in it, uh, but it…

there's just so much of it that
I'd, I'd rather not watch it.

I don't watch WWE.

Um, this is wrestling.

Um, all…

I, you know, I…

as a kid, I called it all-star wrestling.

I, I don't even know where, where this is.

It's scripted, uh, wrestling,
and it's entertainment, and I…

you know, people can enjoy that.

It's fantastic.

They're phenomenal athletes, by the way.

Um, stunning athletes,
but it is a performance.

It's, it's Broadway.

It's, it's scripted.

I d- I, I just don't enjoy that.

I, I don't care who wins, you
know, and all that, and that's

not really the point of it.

It's, it's a scripted performance.

It's not, it's not sort of sport,
and at the center of sport has

to be this level of fairness.

And it's interesting because there was
an earlier game that I was watching, and

I, I've been watching quite a bit of it.

I, uh, it's been really,
really entertaining.

It's been incredibly good sport.

I, uh, I've, I've gra- gained
an appreciation for soccer.

I'm not gonna call it football because
that puts me at some level that I,

I, I, I'm now a, a, a knowledgeable
person on that, and I, I'm, I'm not.

I don't, I don't wanna
be a pretender in that.

But I've r- I've thoroughly
enjoyed it to this point.

There was a game I was watching and,
and England was playing Croatia, and

their superstar, uh, Harry Kane, had
a penalty kick and, um, missed it.

Uh, the goalie saved it.

They did a video review of it, and the
goalie left his spot too early, and that

violation, and so on video review, they
gave him a second kick and he made it.

And so he s- he scored And to
me, that was completely fair.

There's a rule, and, uh, as long as
the rule is reinforced, and if it's

reinforced by video, it, it got it right.

And, you know, you could, you could
argue that that seems, that seems

detailed or something, but it's the rule.

And I, I…

And I think that's related to who
I am as a statistician, that there

are rules in the, uh, with- within
that, and that's completely fine.

Um, and I thought I had
no problem with that.

I don't feel that that's unfair.

There's a set of rules,
and you follow the rules.

And that's why this bothers me, that
he's, the, the striker is not suspended

because there are rules and you follow
the rules, and we are not following

the rules This is related to, to golf.

If you watch a golf tournament, you'll
see many times a golfer is in a, a, a

sort of bad position, but they're standing
on a sprinkler head, or their ball is

in a particular position, there's a
camera in the way, and they are legally

allowed to move their ball, and they
generally end up in a better position.

It's not cheating, it is the rules, and
it's, it's important for every golfer

to completely understand the rules,
and there are officials there that

make sure everybody follows the rules.

Golf is incredibly, um, um,
uh, rule-conscious in fairness.

Cheating in golf, you're
essentially done as a golfer.

I mean, you can't cheat in golf, and
they go out of their way to make sure

to protect the field, and everybody,
your playing, uh, partner should be

reinforcing the rules, and it's, it's
a really important part of golf is,

is sort of fairness, so that Scottie
Scheffler, the number one ranked

player in the world, doesn't get an
advantage over the 500th best player.

They play under the same rules.

I'll watch that.

I enjoy that.

That's sport, and, and that's fantastic.

Um, American baseball now
has video review of pitches.

There's a strike zone, and you
can, you can challenge that.

I, I think it's great.

Getting it right, having, having
the rules, I think it's actually

made the umpires, who are, who are
fantastic at what they do, and they

generally get it right, um, has
made them, have made them better.

But all of this has, uh, has been a
very positive thing because everybody

plays under the same rules within that.

There's a fairness to it
which is critical in sports.

Okay.

So what, what, what does that mean?

I've gone on for 15 minutes about
World Cup soccer and my reaction to it.

Um, I, I, I think as a statistician now,
I don't know whether it's that I became

a statistician because, uh, I, I saw
the rules and, and, and, and part of

it, I almost think it's, it's science.

Or being a scientist and working in
clinical trials make me understand and

appreciate the, the importance of it
to Sports and clinical tri-- uh, to,

to clinical trials that then relate
back to sports, and I have such an

incredibly negative reaction to, um,
removing that suspension, um, uh, in

it, and the lack of fairness to it.

We as statisticians in clinical trials,
fairness are everything in clinical

trials, and it is, it is science.

We are, we are learning
about particular therapies.

Are they safe and effective?

Is A better than B?

There's a lot of parallels between
sports and clinical trials.

If there is undue cheating or
unfairness in a clinical trial,

it means we don't know the answer.

Is A better than B?

Is, is, is, is this drug
safe and effective for a

regulator to make a decision?

Um, and let's leave out of it perhaps
influence in, in those decisions, which we

all as clinical trialists, uh, you know,
abhor, uh, within that, um, uh, within it.

And so there's such a parallel in what
we do in clinical trials to this question

of fairness, and part of why I love it is
you can't fake science, uh, within that.

Let's get out of the aspect that you
can fake data, and there's a great deal

of, of things done in clinical trials
to make sure the data is honest data.

It's reported as collected.

So you could fake it by
presenting data that's not right.

Everybody on the treatment lived.

Everybody on the placebo died.

The drug's a, a miracle.

But, but short of that, you can't fake it.

The data and the results of it,
the, the, the, the statistical

analysis tied to it, demonstration,
demonstrating the likelihood of benefit

of that treatment, uh, within it.

There's this, uh, there, there's this
incredible structure and importance to

things we do in clinical trials to create
this fairness, to answer these questions.

Maybe that's why this sort of reacted--
I reacted so negatively, um, to that.

It's, it's the, the, the same
reaction that I had to the whole

aspect of the acetaminophen
and being the cause of autism.

It's because it was not right.

I'm just, just…

Blatantly, it was a violation of
everything about science that we do.

It was wanting it to be right
rather than it being right within

the rules of science inferences
that we made, uh, uh, within it.

It was not supported
by the data within it.

Uh, vaccines causing autism
are not supported by the data.

There's a want to it.

It is-- It's, it's corruption within
it, and there's such a reaction.

We should all react to that as
clinical trialists because it lowers

everything about other conclusions
that we draw within that setting.

And there is this battle, uh, within
it, um, that, that we continually go on

to, to be able to present this as fair,
to learn the truth about the therapies

out there that we're looking at.

Now

We, we, the, the, this adjudication
of a clinical trial, um,

within it is A better than B.

We run these contests, and there is this
sort of analogy to me of sports, and

maybe it's why I, I, I love what I do.

I love sports.

I love clinical trials.

I, I love seeing the
results of these trials.

I love seeing the
inferences on these trials.

What, what conclusions are we making?

Are we understanding sort of what, what's
fair and how to make these conclusions?

A little bit I'll go back, when
I'm asked to present to non-science

people, students, a- anybody about
statistics, I generally don't go

in and present quantitative things.

My draw…

Two, two exam- two examples I go
back to that I think are just so,

so awesome examples, and they're
very, very simple examples.

One is Sherlock Holmes, and it's--
He, he, uh, he wr- Arthur Conan

Doyle wrote about Sherlock Holmes.

It's a sor- story, Silver Blaze, 1892
short story, where there's a disappearance

of a valuable racehorse within it.

And, um, the Scotland Yard detective
asks, "Is there any other point

to wish to, to which you would
wish to draw my attention?"

And Sherlock Holmes says, "To the curious
incident of the dog in the nighttime."

And the Scotland Yard detective says,
"The dog did nothing in the nighttime."

And Sherlock Holmes says, "That
was the curious incident."

And it was the lack of barking
by a dog that's out there that

would bark if somebody was there.

That w- the lack of information was
the incredible information there,

and that's something as statisticians
that it's so important in what we do.

It's not just the data, it's
how did we get the data?

What data did we not see?

How, uh, you know, was
this prospectively defined?

Was this one of 1,000 things
done and presented to us?

That has very different meaning.

The data are the same, very different
meaning to us making inferences

about something than if it was
the primary analysis in the trial.

These are so critically important
within it, but it can be

presented differently to us.

And so that example of making inferences
from data, I, I feel like is what is

so attractive to me about science,
about what I do as a statistician.

The other example is the…

And this is also a famous example.

Um, by the way, I don't know if this
example, you know what I mean, the whole

sense of this is whether this is a, a tall
tale of, of it, but it's such a brilliant

example, uh, I'll assume it- it's true.

It's a story of Abraham Wald making,
um, looking to reinforce fighter j-

fighter planes in World War II, and
the data being collected were the

location of bullet holes in planes
when they went out and during a battle.

Within it, you, they, the location, you,
they got data on where the holes were

on the plane And the, the fascinating
thing was his conclusion was not to

reinforce where the bullet holes were,
but to reinforce where they're not.

Why is that?

Because of the survivorship bias
that you only see the planes that

survive and come back, and they have
bullet holes in certain locations.

What you don't see are
the planes that go down.

You'd love to see…

The best data would be, let me
see where the location is of the

bullet holes that took planes down.

That's where you wanna reinforce.

You don't see that data.

You see planes that survive.

There's a survivorship bias.

And so it's almost as though where
those bullet holes are didn't take the

plane down, but assuming a uniformity
of bullet holes over the plane, and

there, there's an assumption there,
it's where they're not, uh, within it.

Such a cool example of inference from data
and science, uh, wi- within that scenario.

And I believe these are the kinds
of things in clinical trials.

By the way, this survivorship shows up as
incredibly important thing in making me-

in medical decision-making within that.

And I think as scientists in
clinical trial, not, not…

I, I mean, this is, this is also,
it's not just statisticians.

We, we…

This is something we very much deal
with, but clinical trial scientists,

clinicians, everybody involved in
the enterprise of clinical trials and

medical decision-making, th- this is
such an important part, uh, is making

inferences from the data Within that.

So that's-- It's the joy of,
of, for me, clinical trials and

science and the importance of it.

And there are a lot of parts of, of
clinical trials that, um, i-in some

sense, people maybe think are un-
uh, uh, you know, not appropriate.

Uh, many times, uh, running clinical
trials, sponsors will look at subsets,

look at other endpoints, and see data that
looks really good and believe the result

of that, not understanding the process
of looking at many things is a real, uh,

a part about drawing those conclusions.

You're very likely to find
something positive which lowers

the inferential strength of that.

Doesn't mean it's wrong, but it lowers the
inferential strength of that, uh, in that.

So these post-hoc analyses, i-it's, it's
part of the science of these conclusion.

Survivorship bias absolutely
shows up in clinical trials.

If you're comparing to a natural
history s- database and you have a

clinical trial where people enter
the clinical trial, there are people

in the natural history study that
wouldn't get in the clinical trial.

Maybe they die before
they would get the drug.

And if somebody has the disease and has a
period of time before they are randomized

in the trial, and now they take the drug,
it creates a survivorship bias for people

that took the drug that somebody in the
natural history that had the disease that

died before they'd get the drug would
be in the natural history study, but

they wouldn't be in the clinical trial.

And then the clinical trial, uh, uh,
people, uh, patients do better in

comparison to the natural history study.

Such a clear absolute survivorship bias.

It shows up in many times about
making inferences about data.

It's about fairness i-i-in, in
all of this, and it's-- We, we are

the officers of, of this fairness.

Now interestingly, uh, I, I and, and, uh,
Berry Consultants many times are involved

in non-standard clinical trial stuff.

We enjoy being involved
in hard clinical trials.

In the most sort of standard clinical
trial that you enroll two arms to

a fixed sample size, and you do
an analysis, no interim analysis.

In some sense, the, the, the,
the cleanest data you can have.

Um, and it's awesome.

The, the, these trials are awesome,
but we're typically involved in

adaptive designs, Bayesian analyses.

The pod-- The last episode of, of
the-- In the Interim talked about

bias in the estimate of the treatment
effect when you have futility stopping

or superiority stopping and, and,
and the biased estimate of that.

And by the way, it's, it's very different.

It doesn't mean it's wrong.

It has-- This has influence,
and when we get the result,

we understand the influence of
that, uh, completely within that.

Borrowing from subgroups.

If you borrow from an adult
to a pediatric, uh, patient,

it creates bias in that.

Um, if you borrow from group A to group
B, you use controls from a different

era, uh, in a platform trial within that.

Is there potential bias in that?

Now, all of these things, you know, maybe
it feels strange that I started this off

with this very high level, um, that this
feels unfair that this player is allowed

to play in the game W-within the setting,
but yet now talking about adaptive

designs and Bayesian and all that.

It, it's, it, it's incredibly,
um, important that we can't

keep running those trials.

We miss shots on goal.

There are drugs that are not explored.

We have to be efficient in
patient and time and resources.

Uh, otherwise, we get three
really good answers, and we didn't

ask 100 questions within that.

There's an efficiency to this.

Now, this can be, and it is science.

Bayesian is science.

Adaptive designs is science.

Pre-specification in these settings
is so incredibly important.

Covariate adjustment is science, and
it's incredibly important in that.

It's not cheating to adjust for
covariates within the setting.

You might look at the data as the
raw data of A and B, but a covariate

adjustment gives you a different answer.

Is that cheating?

No.

We understand that as science.

We can have very clean science.

Let me give you another example.

Suppose you were collecting data, and
you were comparing to natural history.

We understand, and I just presented how
there's a survivorship bias in comparing

those natural history patients to
patients in a clinical trial setting.

Does that mean we can't use it, or does
that mean we could potentially adjust

and estimate the survivorship bias?

That can be done in very clean scientific
ways that are honest and, and is

science, and that's where w-we, we
kind of live on this, this border and,

and, and the hard part is that's not
cheating, uh, uh, within the setting.

And so, um, adjusting for it,
understanding the bias and early stopping

and understand making conclusions from
that is still incredibly important, and

it's an incredibly positive thing for
clinical trial science, and that's kind of

the s- the space we live in, and I think
the importance of us sharing these ideas

and presenting this, uh, uh, within it

Now, it's, it's the evolution
of the standard trial that's 300

patients, five years, look at the
data, make a conclusion within that.

This can all be done and is fair.

It's a little bit like the advances
of adding video review to it, of,

of new rules within the setting.

It's different than it used to be, but
it can all be done in a, in a fair way.

There could be an avenue where a red
card is reversed as long as it goes

in the rules of how this is done.

And within this particular setting,
it, it went outside the boundaries.

There isn't even really a
setting for that, uh, within it,

which hence that feels unfair.

So we all as statisticians,
clinicians, clinical trial scientists,

regulators, sites, uh, all of this,
we're all officers of science, uh,

in this, and science is evolving.

We are making this machinery better and
better and better, and it, it improves

the human condition to understand
the truth about different treatments,

the, the efficacy and safety of it.

It's, it's all such an important thing,
and maybe that's why I reacted to that

particular case in, in such a way.

Or maybe it's because, uh, I became a
statistician because I reacted to that.

Uh, I, I, I don't know which
one of those it is, but I, I did

have such a reaction to that.

So everybody out there, keep up
the incredible important work

you do, um, uh, within this.

And until the next time, we
will be here in the interim