Judith: Welcome to Berry's In the Interim podcast, where we explore the
cutting edge of innovative clinical trial design for the pharmaceutical and
medical industries, and so much more.
Let's dive in.
Well, welcome everybody back to In the Interim.
I'm your host, Scott Berry.
We dive into all things clinical trial science, statistics, a wide
range at times here on In the Interim.
Today, I'm gonna do another episode of bringing the world of
sports, what drug developers could learn from the world of sports.
This is a topic we've touched on in other episodes, and I'll come back to that.
I'll, I'll make reference to those.
But I wanna spend more time on the sports part of this, and then I'm
going to show you an analogous problem that comes up in clinical trials.
It's a really important problem in clin- clinical trials, and I
think it's so crystal clear in the world of sports, and it's very fun.
So we've been having the following discussion in our family, and yes, uh,
my family has a number of statisticians.
Uh, my, my wife Tammy is a PhD in finance.
I have two PhD kids, very analytical family, and we love the world of sports.
So when we get together, our discussions are probably a little
bit different, but we've been having the following discussion.
My son Cooper, he's 21 years old, he's a junior at a Division
III college, a baseball team.
He goes to Pitzer College in Claremont, California, and Pomona College and
Pitzer College, they're, they're two colleges combined together to play
Division III sports, and he's on their baseball team, and they're quite good.
They went to the Division III, uh, College World Series a couple
years ago, so they're a very good Division III baseball team.
We've been having the following discussion.
So he-- the Division III baseball team, how would my son Cooper's Division
III baseball team fare if they could play the 1927 New York Yankees?
So this is a, a bit of a global show, so I, I, I, I will
touch on this a little bit.
The 1927 New York Yankees are a famous team in baseball.
Uh, you-- By the way, you can, you...
I think you'll pick up on what's going on here, even if baseball is not your sport.
The 1927 New York Yankees were, were-- They, they even
had the label Murderers Row.
Their top six hitters, in baseball you have nine players that bat in your batting
order, and you rotate through them.
So you go one through nine and then back to one through nine.
Typically, they bat, uh, four or five times in a game.
Their top six hitters were famous, uh, baseball players, two of them Hall
of Famers in Lou Gehrig, and you may have heard of Lou Gehrig as ALS is
referred to as Lou Gehrig's disease.
Was incredible baseball player.
He's in the, the Hall of Fame, and Babe Ruth, who might be the famous, most famous
baseball player of all time for, for, for multiple reasons and, and well-earned,
incredible, uh, baseball player and really brought on the, the advent of hitting
home runs in baseball wasn't something that was really done before Babe Ruth.
He hit more home runs than some teams did in the 1920s.
So that team played in 1927, uh, just we'll, we'll call it a hundred
years, essentially a hundred years ago, and they're one of the most
famous teams in professional baseball.
Now, what does it mean?
Uh, the, the-- this question is you have to spend a lot of time talking
about what do you mean by this?
And what, what our discussion goes to is not if Babe Ruth grew up
and was born in 2004 when my son was born and, and, and lived now.
It's if you could fly a time machine back to 1927, you could put them on the,
the, the time machine and bring them forward today to play a game against
my son's Division III baseball team.
Who would win?
Now, it, it, it, it's-- that, that's what I'm gonna refer to as this time
machine aspect of this, is I'm not, I'm not interested in alternative scenarios
where my son is born in 19- uh, 1904 or, or 1920-- 1906 and he's twenty-one
in 1927 and had to grow up in that era.
Alternatively, what would happen if you could take a time machine and
move the Pomona-Pitzer Division III baseball team to 1927 and they play in
Yankee Stadium with the 1927 equipment?
And of course, if the Yankees move forward to today, they get to use
the aluminum bats that my son's team use, the baseballs, the catcher's
equipment, the gloves, all of that.
They, they, they get to use that equipment.
So the two teams play with the same equipment.
They don't have to use that equipment.
So it's, it's a fair game.
Now, we'll come back to that question.
We'll come back to that.
But that's the sort of question being posed.
Now- Many baseball fans would say, oh my God, they'd get
destroyed by the 1927 Yankees.
But we know something about this era.
So one of, and I wrote a chance column years ago where I presented this,
and I'm fascinated by how different athletes would have done at different
times and why and what that all means.
So Johnny Weissmuller was an an Olympic swimmer, and he won the
1924 and 1928 Olympics in swimming.
He won the 100-meter freestyle gold gold medal in 1924 and 1928.
He's an interesting guy because he became a movie star.
He played Tarzan in the movies and became a a very famous guy and was,
at the time, this good-looking guy,
muscular, strong, was was idolized at the time and was this incredible athlete.
So he was sort of the epitome of this incredible athlete in 1924 and 1928.
In the 100 meters in 1924, his his gold medal time in the 100
meters was 59 seconds flat.
If you took that time and you allowed Tarzan, and I'll apologize for this,
but I'll call Johnny Weissmuller Tarzan.
If you took Tarzan and allowed Tarzan to swim 15 separate Tarzans, so they're
all completely rested up and they and they swim 15 consecutive 100 meters.
So 59, 59, 59 consecutively for 15 100-meter swims, he would swim 1445.
59 seconds, 15 consecutive times.
At the time, by the way, a Swedish swimmer had the Olympic record in the 100 meters.
It was 2115.
So 15 Tarzans would have done 1445 and the Olympic record at the time was 2115, a
50% increase over 15 consecutive Tarzans.
So this incredible athlete is allowed to swim 15 consecutively,
perfectly rested, would swim 1445.
The current Olympic record is 1430.
Bobby Fink, and I may be saying his name wrong, I'm not a big swimming
person, a United States swimmer, has the current Olympic record of 1430.
By the way, the current women's Olympic record is 1520.
They, of course, would destroy the swimmers who had to swim 15 lengths.
They, they, they absolutely destroy them.
Essentially, they would finish, uh, you know, at 1430, and the
record, it was 2115 at the time.
Uh, but you could even take Johnny Weissmuller, Tarzan, and allow him to
swim 15 in a row, and the current swimmers would beat them, would beat that time.
If-- The-- I, I imagine if you said something to somebody at the
time, it would be unimaginable that a human could accomplish that.
But that's a nine...
That's mid-1920s swimmers compared to swimmers nowadays would
destroy swimmers at the time.
We know this is true.
A-and by the way, I'm using that as an example because it's completely
cl- comparable in terms of, uh, timing somebody in a pool at the time.
It, it's, it's translatable.
It's apples to apples.
Another case is just thinking about the hundred-meter run.
And Jesse Owens is, is famous.
He's famous for his, uh, Olympics in Berlin with Adolf Hitler
there, and Adolf Hitler's...
I, I, I don't have to go through this, but Jesse Owens, incredibly
famous for in the face of Adolf Hitler, he won four gold medals.
Uh, this incredible feat.
And in, in the 1936 Olympics, he ran ten point three in the hundred meters.
Now, if you compare that to the, the, the, the Olympic record now is
nine point six three, uh, Usain Bolt.
The-- Essentially, if Usain Bolt at that time would've raced Jesse
Owens, Jesse Owens would be about seven meters behind when Usain Bolt
finishes, uh, essentially destroying Jesse Owens, the, this phenomenal
athlete who accomplished a great deal.
The NCAA record, by the way, now is nine point eight two seconds,
also destroying Jesse Owens.
If you could take Jesse Owens and move him in a time machine from 1936 Berlin to now,
he wouldn't even make these NCAA teams.
Now, the important part of this, a- and I'm gonna, I'm gonna say
this as we go through this, I'm not interested in what they accomplished.
Jesse Owens accomplished an incredible amount, and he was the best runner of his
time, and he should be rewarded for that.
If you're gonna do something like a hall of fame, Jesse Owens is in it.
He's in every hall of fame you should ever think about for track
and field, for every reason.
The fact that he would now not make the, the better NCAA teams, I, I, I'm not even
sure where that fits in with high school, uh, doesn't take away from his accomplish.
But if you could take him in a time machine, that's the truth of it, and
we know this from many other sports.
The shot put is something that hasn't changed, and in the 1920s
they threw it about 50 meters for records, and now they throw it...
Uh, sorry, um, over 50 feet, and now they throw it over 75 feet.
Again, a, a 50% increase in the time.
So the 1920s and now are very, very different.
Uh, and we know this looking at objective measures.
It's much harder in a team sport.
It's much harder thinking about baseball because we don't
have those objective measures.
Looking at a home run in 1927 and a home run for, uh, uh, an NCAA team
now are very, very different things.
Now, what...
By the way, why?
Why are athletes better now?
I think the number one reason, and by the way, I wrote a s- uh, a chance
column on this, uh, 15 Tarzans compared to one modern man, is population.
There are just many, many more people now than there were then,
and if you take the, the most elite of them compared to the most elite
when there's a smaller population, they're largely gonna be better.
Now, you get into extreme value theory and the, the, the chance of
getting this outlier in it, and it can happen in, in parts of different era.
Now, there's no question the world has changed in terms of better health.
Humans are healthier now than then.
It also changes the, the access of somebody.
How many people in 1927 had access to professional baseball that
they would have ha- led a life that gave them access to that?
In 1927, Blacks couldn't have played for the, the, the Yankees
at that time, and, uh, that...
But many people lived in scenarios where even if they had been this
elite athlete- They would have not ended up on the 1927 Yankees.
Now, there's much more access to that from a much wider population base.
If my son would have end up being this elite extreme value, he might
be in the major leagues at this time.
Now, there- so there's better health, there's better access, they're stronger,
they're taller, they're bigger.
The training is absolutely better, strength training
is better, diet is better.
I'll bet my son, Cooper, has played more baseball than those 27 Yankees by
the time they got to, to the Yankees.
Now, they played a ton of baseball, 154 games a year at that time and
all that, but at 21 years old, he's played a lot of baseball.
Played competitive baseball in the youth, and there are many, many people like that.
Video training and all of this, uh, at the time.
So there are many things different.
I, I don't so much care about that.
I care about the time machine part of it.
A direct comparison of the teams or the players across these
eras is, is, is my interest.
Now,
can we address this question of comparing players of different eras?
So I'm interested, what would Babe Ruth do if you could take a time machine, go
back to 1927, you could pull him forward and let him play in 2026 for the Yankees,
uh, batting in front of Aaron Judge.
What would Babe Ruth do?
I'm fascinated by that question.
Fascinated by the question, what would happen if you took Aaron
Judge and you could send him back to 1927 and put him on the Yankees?
Aaron Judge is, like, six foot seven, a, a phenomenal athlete, phenomenal
baseball player, uh, uh, of any era.
What would happen?
So th- th- this is my question.
Now, generally, this, yes, this becomes pub fodder and lots of people discuss,
and generally they say, "You can't compare players of different eras."
But of course we can.
And we can do these models, we can do these estimates,
we can do these comparisons.
So I am, I'm gonna touch on a paper that, uh, I wrote with Shane Reese and Pat
Larkey, and it came out in the American Statistical Association in JASA in 1999.
It was selected as the Applications and Case Studies paper award, so it was-- it
had its own presentation at JSM in, must have been 2000, uh, when that came out.
Uh, when, when that discussion came out where we did exactly that.
Now, how can we compare?
Now, I wanna go back, and I'm gonna give you results from that paper.
Now, that was conducted in 1997, so we did the analyses and the data in 1997.
It would be so much more fun 30 years later to update a lot of
these, and we have plans to do this, but, uh, ha- haven't done...
Some have, and I'll make reference to those.
Uh, Shane Reese has done some updates of that.
In 1997...
Now, now, th- how can we compare those players?
So I'm gonna make reference to Mark McGwire, and Mark McGwire had set a home
run record at that time that crushed Babe Ruth, and he hit 70 home runs,
where Babe Ruth had the re- had, uh, once had the record at 60 home runs.
And people, that's not comparable.
Babe Ruth hitting 60 home runs, I think it was 1924, to Mark McGwire in 1996
hitting 70, you can't compare them.
But Babe Ruth played with players like Jimmie Foxx, and they
overlapped at different times.
Jimmie Foxx was a home run hitter, and Jimmie Foxx played with, I
think he actually even managed Ted Williams, who had a long career.
And so Babe Ruth and Ted Williams never played at the same time, but Jimmie Foxx
played with both of them, and so he's a bridge from Babe Ruth to Ted Williams.
Now, Ted Williams had somewhat of a long career, played, um, uh...
It was incredibly good when he was older.
He overlapped with Hank Aaron, and Hank Aaron had an incredibly long career,
so they played at the same time.
And again, Hank Aaron never played with Babe Ruth.
Uh, Hank Aaron was Black, and Babe Ruth played at a time when Blacks
were not in the Major Leagues.
But there was an overlap of Jimmie Foxx to Ted Williams to Hank Aaron, and then Hank
Aaron overlapped with Reggie Jackson, Mr.
October, who did overlap with Mark McGwire.
There is a bridge of players from Babe Ruth to Mark McGwire, and it's
not just that bridge of four players.
There were thousands of players that overlapped from Babe Ruth all
the way through to Mark McGwire.
This is not a broken, uh, a broken s- uh, graph here.
There's complete overlap or bridging from player to player.
So we fit a model, and you can think of all of these are linear models.
Now, we did three sports, and those sports were baseball, where we modeled
home runs, home run percentage.
That's a, a good outcome in baseball where you hit the ball over the fence,
and you have a certain number of, of attempts and successes, binary data.
Also, a common thing in baseball is how many successful hits you get, not just
home runs, and that's batting average.
And again, that's, uh, a binary outcome of, of success and
failure, so your proportion of those is your batting average.
We did batting average, home run average.
We did NHL scoring, so-- and that's a Poisson distribution
within a game of points.
And we have the data, and I'll say a little bit more
about the data in a minute.
And so when we set up this linear model, it's just a, a log
linear model in the Poisson rate.
And it's a, uh, logistic regression model for baseball and in the, the log odds.
And we did golf.
And golf is...
I'm gonna come back to golf.
Golf is the most pure of these.
Golf, we can do this incredibly well, and I'll come back to some of
the issues in baseball and hockey.
Uh, partly it's that the score-- the, the things we're looking at are
not the full measure of a player, which, which i-is a bit different.
Uh, but golf, there's only one goal in golf, and that's to shoot a low score.
And we have the scores of golfers over time from the nineteen
twenties through now, and we have an incredible amount of overlap in those.
So think of a linear model that's modeling the player.
It's modeling the year.
And that year, embedded in the year is the equipment and the rules and the,
the dynamic of the game at that time.
But we're estimating what that is.
That's an important part, is we estimate the effect of time, and we
can do it because of this overlap.
So the player and the time, and you have this giant linear model across it.
Now, the, the-- And you're sitting there saying, "Aha," but the, the,
the issue you have is players' age And in that we have to model age.
So it's player, it's year, it's age, and in each sport it's a little bit different.
For example, in golf, we model every round in golf because weather, course
difficulty, all of that, we can estimate that incredibly well, and
that's kind of embedded within time.
So each one of these has a, a, some different factors to them that are
relatively straightforward covariates.
But age is critical in this whole aspect of modeling different
players over time in sport.
We modeled that there was a region in the middle of your career where
that's what we're trying to estimate.
And for example, in, in home run hitting, by the way, the optimal year for home
run hitting is twenty-nine years old.
For batting average, it's twenty-seven.
So there's an, a, a region around that, and then we model age.
How do players age in a, a specific young and a specific
old parameter for each player?
So we model even that some players age differently in that.
Tiger Woods in golf has aged differently than, say, a Ben Hogan or a Jack
Nicklaus in that, and he's aged poorly actually for, for well-known reasons.
He- mostly health reasons.
So we can model the age of those players, and when I talk about estimating their
skill, I'm gonna talk about during that time of, of optimal performance.
Now, there are a few players that are a little weird.
Barry Bonds, for example, hit a large number of home runs
when he was a little bit older.
What it essentially sa- uh, estimates is that he aged incredibly well from,
fro- over this time period, uh, in it.
Now, we also model, important part 'cause I'm gonna come back to the populations.
How good were players in the '20s, '30s, '40s, is we have a
random effects model for players from the era in which they come.
We actually did ten-year chunks in that.
I would do it differently now, thirty years later, hopefully.
Uh, hopefully as a statistician, I've aged better, and I would model this better now.
Uh, within it, I would model it really as this sort of, uh, uh, moving
normal dynamic linear model where there we did ten-year chunks in it.
So a player comes from the era in which they were born.
Now, I do wanna say one more thing about the data availability.
When we were doing this in the late '90s, nowhere near the data availability now.
And, um, my, my wife, my, my wife Tammy will remember, we bought a
book called Total Hockey And we sat in a room where she read me
the stats of NHL players over time.
We drew a line and said you had to play a certain number of games.
So she went page by page, she read me the stats and their birth date, and I
typed them in, and we must have spent a month, multiple hours doing this.
Pat Larkey was able to get it-- the, the data on golf.
We ended up using only majors for golf because we just didn't have the data.
Now you can go online, and I think you could, you could ask one of
these large language models to get you this data and the birth date,
and you have it in a, a minute.
You'd have phenomenal data.
In baseball as well, there's incredible resources for base-baseball.
Uh, I...
There were good data for baseball at the time, uh, wi-with ages.
Hockey and golf was much more challenging.
So the data availability, we, we didn't necessarily have it.
We didn't actually...
We couldn't find every golfer and, uh, in that, but much better data now.
Okay.
So just some results.
And interestingly, at the...
Remember we did this in the late nineties, and the best home run
hitter, if you could pull everybody out at their elite, was Mark McGwire.
It said Mark McGwire is the best home run hitter of all time, at that time.
Tony Gwynn was the best batting average of all time, at that time.
The best NHL scorer, rather surprisingly, was Mario Lemieux.
It was not Wayne Gretzky, and I was shocked by that.
And now I won't go in...
I, I don't wanna spend too much detail on that, but largely, Wayne
Gretzky played in an era where there was significantly more scoring.
Now, he contributed to that.
Uh, and Mario Lemieux, they did overlap in time within that, and Mario Lemieux, much
of his career was a time where scoring was, was significantly lower, and yet
he still posted, uh, crazy good numbers.
And the best golfer of all time was Jack Nicklaus.
Interestingly, that was nineteen ninety-seven, and in the dataset
was this youngster, Tiger Woods.
He was in the dataset, and he had won the nineteen ninety-seven
Masters, famously going away, and he was very young at the time.
I don't know, twenty, twenty-one years old maybe.
And, um, and so that was really the only data in there, and the
random effects model shrunk him, and he was number twenty on there.
Shane Reese says, "Rerun this," and says Tiger Woods comes
out as the best of all time.
And, uh, I believe that And it, it's a similar thing to the Jesse Owens
thing, the Johnny Weissmuller thing, for, for all of these, and I'll come
back to the population part of it.
Now, you can go in and look at the paper a- and look at these estimates.
It also gives estimates of their career profile, and there's some
certainly some interesting career profiles where some players, uh, aged
much better, some aged worse, but this is a- at, at their peak time.
And but I'm really interested in how performance changed over time.
So I mentioned hockey and baseball are a little bit challenging, so when we
look at how the population performs for home runs over time, it's a bit
odd because there are some players in baseball who don't want to hit home runs.
Ozzie Smith famously was an incredible defensive player,
and he's in the Hall of Fame.
He didn't hit home runs.
You know, he might hit one or two a year accidentally.
And so if you look at populations, how they behave over time, it's a bit
misleading 'cause he's in there, and he's a quite poor home run hitter.
So you don't wanna compare him to somebody at the other time.
So hockey also, there are players in there who are not trying to score goals.
Their, their job is not to score goals.
They, they're, they...
It's to or, or to assist on goals.
It is to, uh, play defense.
And so hockey is a little bit awkward as well.
Now, the same pattern holds in both those sports despite this, is that
we can look at the estimate of every player in this data set if they
were to all play in the same season.
That's the beauty of this time machine, is you can subtract time effects, and
you can subtract age effects and say, you take at their peak, and they play
at the same time, who's the best?
That was the question I always was after here in that.
So I wanna focus especially on golf because measuring golf is...
There's only one goal in golf, to shoot a low number.
So there is not this players, you know, are really good putters,
but they shoot high numbers, but they're trying to be good putters.
Uh, if you, if you did a scramble or something, it might be there,
but golf is a pure thing at this.
You see this incredible change when you look at the year they were born
and their estimate of if they played all at the same time, you see this
incredible downward Uh, to good scores.
Downward progression of players over time in, in, in two ways you see,
and we plot the median, the 90th percentile, and the 10th percentile.
They all come down sharply from 1920 through...
At this time it was sort of 19, yeah, you know, 65 through 1970
were the years they were born by the time we did this analysis.
There's a sharp decline.
The ni- the 90th percentile, which is on the bad end of
golf, decreases dramatically.
So somebody born in 1930, they're...
The, the 90th percentile was about 76 on this fictional, uh, same golf course.
That 90th percentile for those born in the mid-1960s comes
down to about 73 and a half.
Two shots difference, which is enormous in golf.
The median comes down, and that's about a shot plus different,
which is also enormous.
The 10th percentile goes down with a slightly less slope.
So you see this huge shrinking of players down to the better ones reflective of
there are many, many more players that could be on the PGA Tour and could play
in these majors that weren't there in the 1930s, and it's reflective of this
population all getting down to this.
Now, some of the elite players at those times, as we talked about this
extreme value theory, were there.
And the Ben Hogans and the Sam Sneads, they were great players.
Now, the median player at the time isn't nearly as good as the median
player later within that, and this analysis is reflective of that.
It says exactly that w- in this, in this time machine that
we're able to look at this.
An interesting thing about golf is my brother's in this data set.
So, uh, my brother was born 1961, I think.
Um, and he, um, he played in, uh, six majors, I think.
He played in the 1991 US Open and he played in several PGAs.
He's a professional golfer, and he was never on the PGA Tour.
He went to Q-school, was a very, very good player at the time, but
wasn't good enough to be on the PGA Tour, but was good enough to qualify
for majors and played in majors.
Interestingly, if you'd have taken him and moved him back 30 or 40
years, he would have been better than the median player at those times.
So it's, I, you know, it's, it's this time machine aspect, and we can measure this
incredibly well in golf And by the way, golf is a sport that if we updated this,
I think you'd even see drastically in the last, from the time we did this, in the
last 30 years, because so many players now have access to play golf and to be
in this data set and to be in the elite.
And golf has exploded.
Part of it is money.
The money availability, the training, and all of that.
Yeah, I know that, uh, uh...
You know, I hit the ball farther now than I did when I was younger because
of the equipment, but this is taking the equipment out of it because the,
the round and the time pulls that out.
Okay.
So the population part of that, uh, comes out of that.
Interestingly, when these results came out, it got quite a bit of
press and people interested in that, and I, I wanna reflect on this.
The LPGA Tour actually contacted me, and they were interested.
I didn't run women's golf, and I felt bad all of a sudden when they
contacted me, but they, they've always tried to figure out automatic
qualifications for the Hall of Fame.
And they were interested in whether a model like that could do that
for the Hall of Fame, and it comes back to what I mentioned on this.
This is not about Hall of Fame.
It could say, for example, that all the women that played in the 1960s couldn't
even play on the LB- PGA Tour right now.
Doesn't mean they shouldn't be in the Hall of Fame.
The Hall of Fame should measure relative to your peers, how did you perform?
What did you accomplish in golf?
Otherwise, we have to throw everybody out of the Hall of Fame,
and it's only recent players.
If it's purely objectively, how would that player have done?
Now, the best of the best are much more comparable within that.
By the way, Babe Ruth came out as the third-best home run hitter at that time.
I think if you move that forward now, he probably drops into the teens or
the 20s, that the world of, of home runs has changed dramatically in that.
Uh, but Jack Nicklaus is comparable.
You could take Jack Nicklaus of 1965 and move him now and playing, and
he would be one of the best players.
Canada, of course, cared about hockey and actually did a, a, a call-in show,
uh, from a show in Winnipeg, and the...
A call in...
They, they did.
They, they, they allowed listeners to call in and ask questions of me.
And, and interestingly, they weren't all that worried about the Lemieux-Gretzky
thing because they're both Canadian, and had one of them been a, a US, I think
that, that they would've been upset about that, but they were both Canadian.
But the, the caller said, "What does a Texan know about hockey?"
And I said, "By the way, I grew up in Minneapolis."
I actually grew up playing hockey, um, in, in that.
I grew up in Minneapolis and immediately, "Oh, okay.
That's okay then."
It was so-- So it was sort of didn't matter my statistical chops.
It's where do I know something about hockey?
And, and then another caller came in and said, "Yeah, but your model
can't measure heart, the heart of these players that played back then."
Um, of course, my comment was, "Well, they should have scored more.
They should have used heart to score more, uh, within that."
But hockey again is, is much more complicated to model because,
uh, of the differing goals.
The UK cared about the golf and the US cared about home runs, mostly home runs.
They didn't care so much about batting average.
All right.
So this has now been some 30 plus minutes.
I haven't said a thing about clinical trials, and I did this work before
I ever worked on a clinical trial.
This was-- I was on faculty at Texas A&M, and I did this work
and, um, I was fascinated by this.
I was fascinated by the dynamics of populations in sports.
Still am fascinated by this.
And then I went into designing clinical trials.
In 2000, we started Berry Consultants and started working on this, and soon
after, uh, the following issue came up.
We were involved, and Don Berry was the, the co-PI of the I-SPY 2 trial.
It was a trial in, in neoadjuvant breast cancer where multiple arms would
go into the trial at the same time.
There's a control arm in the trial, and then arms A, B, and C come in, and
it's a phase II trial, so they had as many as 120 patients, and they could
adaptively stop this before that.
And then that arm leaves the trial, but new arms come in.
But there was a constant control over time, and then there are multiple arms.
And it ended up, I believe, 27 or 28 different arms came
in this trial over time.
And at one point, they had to change the control
because there was-- there were new treatments for that subgroup,
that subtype of breast cancer.
They're stratified by H- HER2 status and hormone receptor status, so it
had a basket aspect to the trial.
But within one of these baskets, there was a new control arm.
The beauty of that was that control was actually in the trial as an
e-experimental arm before that, and we w- we're gonna use that as a new
control, but we're also interested in the new arms compared to the old control.
So now we've got arm 21 in the trial and we're trying to compare it to an arm that
was earlier, and they never overlapped.
They didn't play in the trial in the same era.
And the question was, how do we do these analyses for this new arm because
we want to make that comparison, and we want to make a comparison to the
new control, which by the way, was in earlier and then came in later, and
it had a gap in time between that.
This is exactly the same problem.
So platform trials have exactly the same thing, where we're trying to
make comparisons, direct comparisons.
We don't wanna look at the raw data of what an arm did early in this trial to
an-- what an arm did late in the trial.
It's the Babe Ruth, Mark McGwire problem.
It's different eras, but you don't just throw your arms up
and say, "We can't do that."
We can do that incredibly well.
So we instituted the same model that was in that bridging eras of different
sports into the I-SPY 2 trial.
We institute this model in many, many platform trials because it's
exactly this scenario in a-- this new thing called a platform trial
as it is in that sports application.
And we can pull two arms out, and we can make that direct comparison
between those two arms by bridging.
Now, what you want is to have this overlap.
So arm three and arm fifteen weren't in at the same time, but arm three
overlapped with four, um, overlapped with eight, overlapped with eleven,
that overlapped with fifteen, and the other arms in there overlap.
We can estimate time and what time does in the trial, the same thing
we-- as we can estimate in sports.
Now, we, we, we have a, a paper that talks about the Bayesian time
machine, and it does e-exactly this.
Saville, Berry, Berry, Veli and Berry.
So you can look that up if that's of interest in there.
But this exact same modeling.
Now, let's go back for a second and think about what this means.
Let's go back to the sports.
Where does this model maybe break down?
So if things are these additive effects of players, the model's perfect.
So in sports, if there's an interaction, that's when it breaks down.
Now, what is an interaction in sports mean?
So if we're comparing Jack Nicklaus and his rounds in the 1960's to Scottie
Scheffler's rounds in the twenty, uh, 2026
the equipment's very, very different, no question about it.
Golf balls are very, very different.
But we're not, we're not saying Jack has to bring his equipment.
He could play with today's equipment when he was nine-- in 1965.
But suppose the new sand wedges and the new balls they have,
that Jack just can't utilize them the way Scottie Scheffler can.
And-- But yet you didn't have sand wedges of the same, uh, incredible...
That's one thing that's actually changed over time with a golf ball.
That there's this interaction between them, and, um, moving Scottie
Scheffler back would have not just an additive effect, but would have
an interaction because he couldn't play with the wedges of the time.
Now, I think these are strikingly rare i-in, in in sports that these
interactions-- There may be a little bit to a dynamic sport like hockey, where a
player in the 1940s was much smaller, and it may be harder to kind of do the same
things they did where players are larger.
But mostly, you gotta work really, really hard to figure
out interactions in those sports.
And I know you can come up with them, but, uh, I, I want you to think really
hard about whether it, it, it's reality.
Now, in clinical trials, it's the exact same thing.
That's the potential challenge here.
Is when we do this adjustment, are there interactions?
I would say that they're actually less likely in clinical trials
than they were in sports.
By the way, interestingly, the, the beautiful thing about clinical trials
is we don't have the age effect.
You don't have to model how players age in this bridging
part of it in clinical trials.
Drugs don't age within that.
Now, what might an interaction be?
And the, the prime candidate for where there might be interactions are probably
infectious disease, where an old trial had a result relative to placebo,
and now the disease is different.
So the current trial, maybe that treatment wouldn't be as good Even
those I think are, are unlikely.
COVID was the one prime example because the disease morphed so
quickly and changed so much.
Now, most of the hospitalized trials we did in COVID, it's about the host, and
the host is sick, and it's not really about the, the, the viral-- virus anymore.
But even in that case, it's not clear that there are interactions.
And by the way, if we think there are interactions where treatment was in a
trial before an overlap placebo, and now we have this overlap of arms, and
we think that the relative differences of the arms could switch, we can't
do non-inferiority trials anymore.
And actually, when a trial's over, I'm-- I don't even know if we can think does the
drug work because right now it might have switched with placebo, uh, within that,
and do we know if the treatment works?
So if you're looking at...
And, and what's really interesting, we can test this because of an arm's length of
its career in a trial and the different arms, we can look to see whether it
looks like it varies relative to control.
In I-SPY 2, if you took the same treatment in there at different
times, it was incredibly stable.
Neoadjuvant breast cancer, whether this is ALS, whether this is, um, a
cardiovascular disease, weight loss, this is, um, uh, many other diseases,
it's really hard to think about interactions within those trials.
But that would be the same way in which in a platform trial you would
be concerned about using this model to compare treatments that are in the trial
at different times and different eras.
Okay.
Well,
uh, by the way, if you're, if you're interested in this, you can jump
to episodes twenty and twenty-one, talk about I-SPY 2 with, with Don.
Um, a incredible story in and of itself, but part of that
talks about the time machine.
And then episode twenty-two talks about the time machine much more
from a statistical standpoint with, uh, co-host Kurt Vile.
So those may be of interest to you.
So coming back to the original lead-in to this, uh, Cooper's
team against the 1927 Yankees.
By the way, we don't have the data to make that bridging because
we don't have Division III teams playing pro teams and even Division
I teams playing pro teams in that.
Uh, though, though my son's team did scrimmage, uh, this year a, um,
a rookie league professional team.
Um, in college hockey I think it's clear a college hockey team
now would beat a 1920s NHL team.
E- And yes, they would use the same equipment, so the, that NHL team could
get a little bit of time to practice and use the equipment and all of that.
So it, it, you know, seamlessly that, I, I think a college hockey team now
would beat a team back then for sure.
I think an NCAA Division I baseball team now beats the 1927 Yankees.
I think it, it's just the world has moved on, the, the speed of these pitchers.
I don't know if any 1927 Yankees threw 90 miles an hour.
I don't think they did.
I think a D1 team would, would beat up the 1927 Yankees.
And in golf, it's absolutely clear.
NCAA golfers now are better than the, the, the 1920s.
I, I know there's Bobby Jones and other players like that, but
they would, I think they would, quote-unquote, "beat up" on 1927 golfers.
So where does that leave Division III baseball team now against the '27 Yankees?
I don't know the answer to that.
I think it would actually be a really good game.
And maybe Babe Ruth calls his shot and hits the winning home
run, uh, in a game like that.
But I actually think it would be a reasonable competition.
All right.
Well, um, this is coming to you through a time machine, by the way.
It's recorded, and through time you got to hear this through a time
machine about The Time Machine.
And next time, the next episode may be better because of, of, uh, era effects.
But until the next time, we'll be here in the interim