WEBVTT NOTE This file was generated by Descript 00:00:00.000 --> 00:00:03.529 Judith: Welcome to Berry's In the Interim podcast, where we explore the 00:00:03.529 --> 00:00:07.929 cutting edge of innovative clinical trial design for the pharmaceutical and 00:00:07.929 --> 00:00:10.310 medical industries, and so much more. 00:00:11.029 --> 00:00:11.729 Let's dive in. 00:00:13.328 --> 00:00:16.958 Well, welcome everybody back to In the Interim. 00:00:17.158 --> 00:00:18.728 I'm your host, Scott Berry. 00:00:19.748 --> 00:00:26.808 We dive into all things clinical trial science, statistics, a wide 00:00:26.808 --> 00:00:29.458 range at times here on In the Interim. 00:00:30.198 --> 00:00:34.828 Today, I'm gonna do another episode of bringing the world of 00:00:34.828 --> 00:00:39.078 sports, what drug developers could learn from the world of sports. 00:00:39.848 --> 00:00:44.058 This is a topic we've touched on in other episodes, and I'll come back to that. 00:00:44.058 --> 00:00:45.598 I'll, I'll make reference to those. 00:00:46.138 --> 00:00:50.088 But I wanna spend more time on the sports part of this, and then I'm 00:00:50.088 --> 00:00:55.038 going to show you an analogous problem that comes up in clinical trials. 00:00:55.038 --> 00:00:58.508 It's a really important problem in clin- clinical trials, and I 00:00:58.508 --> 00:01:03.688 think it's so crystal clear in the world of sports, and it's very fun. 00:01:05.028 --> 00:01:10.208 So we've been having the following discussion in our family, and yes, uh, 00:01:10.208 --> 00:01:13.628 my family has a number of statisticians. 00:01:13.988 --> 00:01:17.838 Uh, my, my wife Tammy is a PhD in finance. 00:01:17.838 --> 00:01:23.098 I have two PhD kids, very analytical family, and we love the world of sports. 00:01:23.098 --> 00:01:26.668 So when we get together, our discussions are probably a little 00:01:26.668 --> 00:01:29.318 bit different, but we've been having the following discussion. 00:01:30.368 --> 00:01:35.928 My son Cooper, he's 21 years old, he's a junior at a Division 00:01:35.988 --> 00:01:39.428 III college, a baseball team. 00:01:39.848 --> 00:01:46.108 He goes to Pitzer College in Claremont, California, and Pomona College and 00:01:46.108 --> 00:01:50.678 Pitzer College, they're, they're two colleges combined together to play 00:01:50.678 --> 00:01:55.528 Division III sports, and he's on their baseball team, and they're quite good. 00:01:55.908 --> 00:02:00.508 They went to the Division III, uh, College World Series a couple 00:02:00.508 --> 00:02:05.308 years ago, so they're a very good Division III baseball team. 00:02:06.128 --> 00:02:08.348 We've been having the following discussion. 00:02:08.348 --> 00:02:16.738 So he-- the Division III baseball team, how would my son Cooper's Division 00:02:16.738 --> 00:02:24.178 III baseball team fare if they could play the 1927 New York Yankees? 00:02:25.368 --> 00:02:29.478 So this is a, a bit of a global show, so I, I, I, I will 00:02:29.478 --> 00:02:30.828 touch on this a little bit. 00:02:30.828 --> 00:02:34.938 The 1927 New York Yankees are a famous team in baseball. 00:02:35.398 --> 00:02:37.368 Uh, you-- By the way, you can, you... 00:02:37.528 --> 00:02:42.078 I think you'll pick up on what's going on here, even if baseball is not your sport. 00:02:43.228 --> 00:02:47.348 The 1927 New York Yankees were, were-- They, they even 00:02:47.348 --> 00:02:49.948 had the label Murderers Row. 00:02:50.228 --> 00:02:55.778 Their top six hitters, in baseball you have nine players that bat in your batting 00:02:55.778 --> 00:02:57.308 order, and you rotate through them. 00:02:57.998 --> 00:03:00.998 So you go one through nine and then back to one through nine. 00:03:01.418 --> 00:03:04.873 Typically, they bat, uh, four or five times in a game. 00:03:05.703 --> 00:03:12.203 Their top six hitters were famous, uh, baseball players, two of them Hall 00:03:12.203 --> 00:03:17.373 of Famers in Lou Gehrig, and you may have heard of Lou Gehrig as ALS is 00:03:17.373 --> 00:03:19.313 referred to as Lou Gehrig's disease. 00:03:19.743 --> 00:03:21.713 Was incredible baseball player. 00:03:21.713 --> 00:03:26.843 He's in the, the Hall of Fame, and Babe Ruth, who might be the famous, most famous 00:03:26.843 --> 00:03:31.623 baseball player of all time for, for, for multiple reasons and, and well-earned, 00:03:31.623 --> 00:03:38.473 incredible, uh, baseball player and really brought on the, the advent of hitting 00:03:38.473 --> 00:03:42.773 home runs in baseball wasn't something that was really done before Babe Ruth. 00:03:42.973 --> 00:03:47.803 He hit more home runs than some teams did in the 1920s. 00:03:49.153 --> 00:03:56.113 So that team played in 1927, uh, just we'll, we'll call it a hundred 00:03:56.113 --> 00:03:59.683 years, essentially a hundred years ago, and they're one of the most 00:03:59.683 --> 00:04:02.303 famous teams in professional baseball. 00:04:03.793 --> 00:04:05.183 Now, what does it mean? 00:04:05.413 --> 00:04:09.903 Uh, the, the-- this question is you have to spend a lot of time talking 00:04:09.903 --> 00:04:11.193 about what do you mean by this? 00:04:11.783 --> 00:04:18.423 And what, what our discussion goes to is not if Babe Ruth grew up 00:04:18.423 --> 00:04:24.143 and was born in 2004 when my son was born and, and, and lived now. 00:04:24.363 --> 00:04:32.253 It's if you could fly a time machine back to 1927, you could put them on the, 00:04:32.423 --> 00:04:39.153 the, the time machine and bring them forward today to play a game against 00:04:39.153 --> 00:04:42.633 my son's Division III baseball team. 00:04:43.353 --> 00:04:44.393 Who would win? 00:04:45.483 --> 00:04:51.643 Now, it, it, it, it's-- that, that's what I'm gonna refer to as this time 00:04:51.643 --> 00:04:59.603 machine aspect of this, is I'm not, I'm not interested in alternative scenarios 00:04:59.603 --> 00:05:12.203 where my son is born in 19- uh, 1904 or, or 1920-- 1906 and he's twenty-one 00:05:12.203 --> 00:05:15.113 in 1927 and had to grow up in that era. 00:05:15.723 --> 00:05:19.223 Alternatively, what would happen if you could take a time machine and 00:05:19.223 --> 00:05:26.073 move the Pomona-Pitzer Division III baseball team to 1927 and they play in 00:05:26.073 --> 00:05:30.963 Yankee Stadium with the 1927 equipment? 00:05:30.963 --> 00:05:36.623 And of course, if the Yankees move forward to today, they get to use 00:05:37.363 --> 00:05:41.943 the aluminum bats that my son's team use, the baseballs, the catcher's 00:05:41.943 --> 00:05:44.543 equipment, the gloves, all of that. 00:05:44.653 --> 00:05:46.543 They, they, they get to use that equipment. 00:05:46.543 --> 00:05:48.853 So the two teams play with the same equipment. 00:05:48.853 --> 00:05:50.303 They don't have to use that equipment. 00:05:50.303 --> 00:05:52.393 So it's, it's a fair game. 00:05:53.453 --> 00:05:55.353 Now, we'll come back to that question. 00:05:55.353 --> 00:05:56.583 We'll come back to that. 00:05:58.593 --> 00:06:01.753 But that's the sort of question being posed. 00:06:03.083 --> 00:06:08.173 Now- Many baseball fans would say, oh my God, they'd get 00:06:08.173 --> 00:06:10.663 destroyed by the 1927 Yankees. 00:06:11.193 --> 00:06:14.443 But we know something about this era. 00:06:14.443 --> 00:06:23.133 So one of, and I wrote a chance column years ago where I presented this, 00:06:23.133 --> 00:06:29.233 and I'm fascinated by how different athletes would have done at different 00:06:29.233 --> 00:06:33.423 times and why and what that all means. 00:06:33.843 --> 00:06:41.663 So Johnny Weissmuller was an an Olympic swimmer, and he won the 00:06:41.663 --> 00:06:47.013 1924 and 1928 Olympics in swimming. 00:06:47.403 --> 00:06:54.213 He won the 100-meter freestyle gold gold medal in 1924 and 1928. 00:06:55.483 --> 00:06:59.673 He's an interesting guy because he became a movie star. 00:07:00.163 --> 00:07:05.543 He played Tarzan in the movies and became a a very famous guy and was, 00:07:05.833 --> 00:07:09.923 at the time, this good-looking guy, 00:07:12.203 --> 00:07:19.323 muscular, strong, was was idolized at the time and was this incredible athlete. 00:07:19.383 --> 00:07:25.263 So he was sort of the epitome of this incredible athlete in 1924 and 1928. 00:07:25.753 --> 00:07:34.053 In the 100 meters in 1924, his his gold medal time in the 100 00:07:34.053 --> 00:07:36.823 meters was 59 seconds flat. 00:07:39.273 --> 00:07:48.663 If you took that time and you allowed Tarzan, and I'll apologize for this, 00:07:48.663 --> 00:07:50.773 but I'll call Johnny Weissmuller Tarzan. 00:07:51.063 --> 00:07:58.213 If you took Tarzan and allowed Tarzan to swim 15 separate Tarzans, so they're 00:07:58.263 --> 00:08:06.953 all completely rested up and they and they swim 15 consecutive 100 meters. 00:08:07.463 --> 00:08:18.563 So 59, 59, 59 consecutively for 15 100-meter swims, he would swim 1445. 00:08:19.813 --> 00:08:24.373 59 seconds, 15 consecutive times. 00:08:25.723 --> 00:08:33.843 At the time, by the way, a Swedish swimmer had the Olympic record in the 100 meters. 00:08:34.063 --> 00:08:36.033 It was 2115. 00:08:36.893 --> 00:08:45.593 So 15 Tarzans would have done 1445 and the Olympic record at the time was 2115, a 00:08:45.593 --> 00:08:49.903 50% increase over 15 consecutive Tarzans. 00:08:49.903 --> 00:08:56.533 So this incredible athlete is allowed to swim 15 consecutively, 00:08:56.533 --> 00:08:59.193 perfectly rested, would swim 1445. 00:09:02.158 --> 00:09:05.088 The current Olympic record is 1430. 00:09:05.878 --> 00:09:09.528 Bobby Fink, and I may be saying his name wrong, I'm not a big swimming 00:09:09.528 --> 00:09:15.418 person, a United States swimmer, has the current Olympic record of 1430. 00:09:15.418 --> 00:09:19.478 By the way, the current women's Olympic record is 1520. 00:09:20.178 --> 00:09:27.148 They, of course, would destroy the swimmers who had to swim 15 lengths. 00:09:27.728 --> 00:09:30.218 They, they, they absolutely destroy them. 00:09:30.218 --> 00:09:35.318 Essentially, they would finish, uh, you know, at 1430, and the 00:09:35.318 --> 00:09:38.088 record, it was 2115 at the time. 00:09:38.678 --> 00:09:43.748 Uh, but you could even take Johnny Weissmuller, Tarzan, and allow him to 00:09:43.748 --> 00:09:48.878 swim 15 in a row, and the current swimmers would beat them, would beat that time. 00:09:49.158 --> 00:09:53.058 If-- The-- I, I imagine if you said something to somebody at the 00:09:53.058 --> 00:09:58.958 time, it would be unimaginable that a human could accomplish that. 00:10:00.098 --> 00:10:00.968 But that's a nine... 00:10:01.058 --> 00:10:06.768 That's mid-1920s swimmers compared to swimmers nowadays would 00:10:06.768 --> 00:10:08.388 destroy swimmers at the time. 00:10:11.178 --> 00:10:12.258 We know this is true. 00:10:12.258 --> 00:10:16.928 A-and by the way, I'm using that as an example because it's completely 00:10:17.058 --> 00:10:24.928 cl- comparable in terms of, uh, timing somebody in a pool at the time. 00:10:25.258 --> 00:10:27.168 It, it's, it's translatable. 00:10:27.168 --> 00:10:28.438 It's apples to apples. 00:10:28.888 --> 00:10:32.048 Another case is just thinking about the hundred-meter run. 00:10:32.888 --> 00:10:36.148 And Jesse Owens is, is famous. 00:10:36.228 --> 00:10:43.468 He's famous for his, uh, Olympics in Berlin with Adolf Hitler 00:10:43.468 --> 00:10:45.828 there, and Adolf Hitler's... 00:10:45.998 --> 00:10:50.098 I, I, I don't have to go through this, but Jesse Owens, incredibly 00:10:50.098 --> 00:10:54.128 famous for in the face of Adolf Hitler, he won four gold medals. 00:10:54.828 --> 00:10:56.898 Uh, this incredible feat. 00:10:57.278 --> 00:11:07.628 And in, in the 1936 Olympics, he ran ten point three in the hundred meters. 00:11:09.038 --> 00:11:13.088 Now, if you compare that to the, the, the, the Olympic record now is 00:11:13.088 --> 00:11:16.498 nine point six three, uh, Usain Bolt. 00:11:17.578 --> 00:11:22.578 The-- Essentially, if Usain Bolt at that time would've raced Jesse 00:11:22.578 --> 00:11:27.698 Owens, Jesse Owens would be about seven meters behind when Usain Bolt 00:11:27.698 --> 00:11:34.698 finishes, uh, essentially destroying Jesse Owens, the, this phenomenal 00:11:34.698 --> 00:11:37.078 athlete who accomplished a great deal. 00:11:37.368 --> 00:11:42.088 The NCAA record, by the way, now is nine point eight two seconds, 00:11:42.178 --> 00:11:44.828 also destroying Jesse Owens. 00:11:45.368 --> 00:11:53.698 If you could take Jesse Owens and move him in a time machine from 1936 Berlin to now, 00:11:53.918 --> 00:11:58.678 he wouldn't even make these NCAA teams. 00:12:00.103 --> 00:12:03.943 Now, the important part of this, a- and I'm gonna, I'm gonna say 00:12:03.943 --> 00:12:09.243 this as we go through this, I'm not interested in what they accomplished. 00:12:09.683 --> 00:12:16.423 Jesse Owens accomplished an incredible amount, and he was the best runner of his 00:12:16.423 --> 00:12:19.713 time, and he should be rewarded for that. 00:12:19.763 --> 00:12:23.303 If you're gonna do something like a hall of fame, Jesse Owens is in it. 00:12:23.773 --> 00:12:27.013 He's in every hall of fame you should ever think about for track 00:12:27.013 --> 00:12:28.863 and field, for every reason. 00:12:29.333 --> 00:12:37.443 The fact that he would now not make the, the better NCAA teams, I, I, I'm not even 00:12:37.443 --> 00:12:43.543 sure where that fits in with high school, uh, doesn't take away from his accomplish. 00:12:43.633 --> 00:12:48.593 But if you could take him in a time machine, that's the truth of it, and 00:12:48.593 --> 00:12:50.433 we know this from many other sports. 00:12:50.433 --> 00:12:55.773 The shot put is something that hasn't changed, and in the 1920s 00:12:55.773 --> 00:12:59.413 they threw it about 50 meters for records, and now they throw it... 00:12:59.413 --> 00:13:05.863 Uh, sorry, um, over 50 feet, and now they throw it over 75 feet. 00:13:05.953 --> 00:13:08.913 Again, a, a 50% increase in the time. 00:13:09.053 --> 00:13:12.453 So the 1920s and now are very, very different. 00:13:13.493 --> 00:13:16.103 Uh, and we know this looking at objective measures. 00:13:16.343 --> 00:13:18.263 It's much harder in a team sport. 00:13:19.183 --> 00:13:23.063 It's much harder thinking about baseball because we don't 00:13:23.063 --> 00:13:24.823 have those objective measures. 00:13:24.853 --> 00:13:30.713 Looking at a home run in 1927 and a home run for, uh, uh, an NCAA team 00:13:30.713 --> 00:13:32.603 now are very, very different things. 00:13:33.403 --> 00:13:33.903 Now, what... 00:13:33.973 --> 00:13:34.953 By the way, why? 00:13:35.053 --> 00:13:39.183 Why are athletes better now? 00:13:40.813 --> 00:13:45.213 I think the number one reason, and by the way, I wrote a s- uh, a chance 00:13:45.213 --> 00:13:52.013 column on this, uh, 15 Tarzans compared to one modern man, is population. 00:13:52.893 --> 00:13:57.623 There are just many, many more people now than there were then, 00:13:57.623 --> 00:14:03.813 and if you take the, the most elite of them compared to the most elite 00:14:03.813 --> 00:14:06.773 when there's a smaller population, they're largely gonna be better. 00:14:07.503 --> 00:14:11.303 Now, you get into extreme value theory and the, the, the chance of 00:14:11.303 --> 00:14:15.873 getting this outlier in it, and it can happen in, in parts of different era. 00:14:16.433 --> 00:14:20.683 Now, there's no question the world has changed in terms of better health. 00:14:21.193 --> 00:14:23.923 Humans are healthier now than then. 00:14:24.363 --> 00:14:28.223 It also changes the, the access of somebody. 00:14:28.223 --> 00:14:35.523 How many people in 1927 had access to professional baseball that 00:14:35.523 --> 00:14:38.883 they would have ha- led a life that gave them access to that? 00:14:38.883 --> 00:14:44.043 In 1927, Blacks couldn't have played for the, the, the Yankees 00:14:44.333 --> 00:14:47.093 at that time, and, uh, that... 00:14:47.093 --> 00:14:51.673 But many people lived in scenarios where even if they had been this 00:14:51.673 --> 00:14:56.968 elite athlete- They would have not ended up on the 1927 Yankees. 00:14:57.228 --> 00:15:02.588 Now, there's much more access to that from a much wider population base. 00:15:02.988 --> 00:15:10.708 If my son would have end up being this elite extreme value, he might 00:15:10.708 --> 00:15:12.288 be in the major leagues at this time. 00:15:13.108 --> 00:15:18.518 Now, there- so there's better health, there's better access, they're stronger, 00:15:19.048 --> 00:15:20.558 they're taller, they're bigger. 00:15:21.018 --> 00:15:24.458 The training is absolutely better, strength training 00:15:24.458 --> 00:15:26.298 is better, diet is better. 00:15:27.218 --> 00:15:32.998 I'll bet my son, Cooper, has played more baseball than those 27 Yankees by 00:15:32.998 --> 00:15:35.098 the time they got to, to the Yankees. 00:15:35.098 --> 00:15:39.108 Now, they played a ton of baseball, 154 games a year at that time and 00:15:39.108 --> 00:15:43.088 all that, but at 21 years old, he's played a lot of baseball. 00:15:43.348 --> 00:15:47.478 Played competitive baseball in the youth, and there are many, many people like that. 00:15:47.478 --> 00:15:51.128 Video training and all of this, uh, at the time. 00:15:51.128 --> 00:15:52.978 So there are many things different. 00:15:53.738 --> 00:15:56.158 I, I don't so much care about that. 00:15:56.188 --> 00:15:59.018 I care about the time machine part of it. 00:15:59.498 --> 00:16:04.818 A direct comparison of the teams or the players across these 00:16:04.818 --> 00:16:06.728 eras is, is, is my interest. 00:16:07.858 --> 00:16:08.278 Now, 00:16:10.538 --> 00:16:16.408 can we address this question of comparing players of different eras? 00:16:16.408 --> 00:16:23.218 So I'm interested, what would Babe Ruth do if you could take a time machine, go 00:16:23.218 --> 00:16:31.238 back to 1927, you could pull him forward and let him play in 2026 for the Yankees, 00:16:31.898 --> 00:16:34.298 uh, batting in front of Aaron Judge. 00:16:34.328 --> 00:16:35.608 What would Babe Ruth do? 00:16:36.648 --> 00:16:38.758 I'm fascinated by that question. 00:16:39.098 --> 00:16:42.398 Fascinated by the question, what would happen if you took Aaron 00:16:42.398 --> 00:16:48.038 Judge and you could send him back to 1927 and put him on the Yankees? 00:16:48.438 --> 00:16:53.648 Aaron Judge is, like, six foot seven, a, a phenomenal athlete, phenomenal 00:16:53.648 --> 00:16:55.878 baseball player, uh, uh, of any era. 00:16:55.878 --> 00:16:56.518 What would happen? 00:16:56.558 --> 00:16:58.308 So th- th- this is my question. 00:16:58.768 --> 00:17:03.288 Now, generally, this, yes, this becomes pub fodder and lots of people discuss, 00:17:03.288 --> 00:17:06.678 and generally they say, "You can't compare players of different eras." 00:17:08.358 --> 00:17:11.548 But of course we can. 00:17:12.288 --> 00:17:15.928 And we can do these models, we can do these estimates, 00:17:16.208 --> 00:17:18.288 we can do these comparisons. 00:17:18.908 --> 00:17:26.108 So I am, I'm gonna touch on a paper that, uh, I wrote with Shane Reese and Pat 00:17:26.108 --> 00:17:34.498 Larkey, and it came out in the American Statistical Association in JASA in 1999. 00:17:34.568 --> 00:17:40.838 It was selected as the Applications and Case Studies paper award, so it was-- it 00:17:40.838 --> 00:17:47.773 had its own presentation at JSM in, must have been 2000, uh, when that came out. 00:17:48.693 --> 00:17:52.493 Uh, when, when that discussion came out where we did exactly that. 00:17:52.583 --> 00:17:55.083 Now, how can we compare? 00:17:55.083 --> 00:17:59.073 Now, I wanna go back, and I'm gonna give you results from that paper. 00:17:59.673 --> 00:18:06.653 Now, that was conducted in 1997, so we did the analyses and the data in 1997. 00:18:07.393 --> 00:18:12.153 It would be so much more fun 30 years later to update a lot of 00:18:12.153 --> 00:18:16.303 these, and we have plans to do this, but, uh, ha- haven't done... 00:18:16.303 --> 00:18:18.273 Some have, and I'll make reference to those. 00:18:18.663 --> 00:18:21.453 Uh, Shane Reese has done some updates of that. 00:18:22.373 --> 00:18:23.613 In 1997... 00:18:23.613 --> 00:18:27.663 Now, now, th- how can we compare those players? 00:18:27.663 --> 00:18:33.353 So I'm gonna make reference to Mark McGwire, and Mark McGwire had set a home 00:18:33.353 --> 00:18:39.483 run record at that time that crushed Babe Ruth, and he hit 70 home runs, 00:18:39.483 --> 00:18:44.793 where Babe Ruth had the re- had, uh, once had the record at 60 home runs. 00:18:45.593 --> 00:18:47.683 And people, that's not comparable. 00:18:47.973 --> 00:18:54.523 Babe Ruth hitting 60 home runs, I think it was 1924, to Mark McGwire in 1996 00:18:54.523 --> 00:18:56.073 hitting 70, you can't compare them. 00:18:57.523 --> 00:19:05.433 But Babe Ruth played with players like Jimmie Foxx, and they 00:19:05.433 --> 00:19:06.983 overlapped at different times. 00:19:06.983 --> 00:19:12.143 Jimmie Foxx was a home run hitter, and Jimmie Foxx played with, I 00:19:12.643 --> 00:19:16.623 think he actually even managed Ted Williams, who had a long career. 00:19:17.153 --> 00:19:25.453 And so Babe Ruth and Ted Williams never played at the same time, but Jimmie Foxx 00:19:25.453 --> 00:19:33.003 played with both of them, and so he's a bridge from Babe Ruth to Ted Williams. 00:19:34.673 --> 00:19:38.853 Now, Ted Williams had somewhat of a long career, played, um, uh... 00:19:38.853 --> 00:19:41.803 It was incredibly good when he was older. 00:19:41.833 --> 00:19:47.603 He overlapped with Hank Aaron, and Hank Aaron had an incredibly long career, 00:19:47.603 --> 00:19:49.243 so they played at the same time. 00:19:49.243 --> 00:19:51.993 And again, Hank Aaron never played with Babe Ruth. 00:19:52.713 --> 00:19:56.523 Uh, Hank Aaron was Black, and Babe Ruth played at a time when Blacks 00:19:56.523 --> 00:19:58.413 were not in the Major Leagues. 00:19:59.043 --> 00:20:04.703 But there was an overlap of Jimmie Foxx to Ted Williams to Hank Aaron, and then Hank 00:20:04.703 --> 00:20:08.343 Aaron overlapped with Reggie Jackson, Mr. 00:20:08.343 --> 00:20:11.903 October, who did overlap with Mark McGwire. 00:20:12.673 --> 00:20:17.423 There is a bridge of players from Babe Ruth to Mark McGwire, and it's 00:20:17.423 --> 00:20:20.363 not just that bridge of four players. 00:20:20.813 --> 00:20:27.903 There were thousands of players that overlapped from Babe Ruth all 00:20:27.903 --> 00:20:29.333 the way through to Mark McGwire. 00:20:31.223 --> 00:20:38.403 This is not a broken, uh, a broken s- uh, graph here. 00:20:38.773 --> 00:20:44.303 There's complete overlap or bridging from player to player. 00:20:45.423 --> 00:20:53.053 So we fit a model, and you can think of all of these are linear models. 00:20:53.053 --> 00:21:01.303 Now, we did three sports, and those sports were baseball, where we modeled 00:21:01.353 --> 00:21:03.933 home runs, home run percentage. 00:21:04.263 --> 00:21:08.103 That's a, a good outcome in baseball where you hit the ball over the fence, 00:21:08.413 --> 00:21:14.253 and you have a certain number of, of attempts and successes, binary data. 00:21:15.563 --> 00:21:20.433 Also, a common thing in baseball is how many successful hits you get, not just 00:21:20.433 --> 00:21:22.983 home runs, and that's batting average. 00:21:23.313 --> 00:21:27.753 And again, that's, uh, a binary outcome of, of success and 00:21:27.753 --> 00:21:31.283 failure, so your proportion of those is your batting average. 00:21:31.823 --> 00:21:35.993 We did batting average, home run average. 00:21:36.773 --> 00:21:42.993 We did NHL scoring, so-- and that's a Poisson distribution 00:21:43.063 --> 00:21:45.423 within a game of points. 00:21:48.113 --> 00:21:50.543 And we have the data, and I'll say a little bit more 00:21:50.543 --> 00:21:51.683 about the data in a minute. 00:21:51.683 --> 00:21:55.893 And so when we set up this linear model, it's just a, a log 00:21:55.893 --> 00:21:58.393 linear model in the Poisson rate. 00:21:59.353 --> 00:22:07.513 And it's a, uh, logistic regression model for baseball and in the, the log odds. 00:22:08.173 --> 00:22:09.623 And we did golf. 00:22:10.863 --> 00:22:12.793 And golf is... 00:22:12.793 --> 00:22:14.413 I'm gonna come back to golf. 00:22:14.413 --> 00:22:16.753 Golf is the most pure of these. 00:22:17.403 --> 00:22:21.063 Golf, we can do this incredibly well, and I'll come back to some of 00:22:21.063 --> 00:22:22.803 the issues in baseball and hockey. 00:22:23.213 --> 00:22:26.603 Uh, partly it's that the score-- the, the things we're looking at are 00:22:26.603 --> 00:22:31.053 not the full measure of a player, which, which i-is a bit different. 00:22:31.573 --> 00:22:36.013 Uh, but golf, there's only one goal in golf, and that's to shoot a low score. 00:22:36.663 --> 00:22:40.983 And we have the scores of golfers over time from the nineteen 00:22:40.983 --> 00:22:47.413 twenties through now, and we have an incredible amount of overlap in those. 00:22:48.143 --> 00:22:52.553 So think of a linear model that's modeling the player. 00:22:53.913 --> 00:22:55.613 It's modeling the year. 00:22:56.943 --> 00:23:02.723 And that year, embedded in the year is the equipment and the rules and the, 00:23:02.773 --> 00:23:05.233 the dynamic of the game at that time. 00:23:05.263 --> 00:23:07.573 But we're estimating what that is. 00:23:08.183 --> 00:23:12.403 That's an important part, is we estimate the effect of time, and we 00:23:12.403 --> 00:23:14.663 can do it because of this overlap. 00:23:15.573 --> 00:23:22.393 So the player and the time, and you have this giant linear model across it. 00:23:23.373 --> 00:23:28.343 Now, the, the-- And you're sitting there saying, "Aha," but the, the, 00:23:28.343 --> 00:23:36.278 the issue you have is players' age And in that we have to model age. 00:23:36.508 --> 00:23:41.008 So it's player, it's year, it's age, and in each sport it's a little bit different. 00:23:41.008 --> 00:23:47.048 For example, in golf, we model every round in golf because weather, course 00:23:47.048 --> 00:23:51.148 difficulty, all of that, we can estimate that incredibly well, and 00:23:51.148 --> 00:23:53.248 that's kind of embedded within time. 00:23:53.248 --> 00:23:57.548 So each one of these has a, a, some different factors to them that are 00:23:57.548 --> 00:23:59.778 relatively straightforward covariates. 00:24:00.108 --> 00:24:06.108 But age is critical in this whole aspect of modeling different 00:24:06.108 --> 00:24:08.718 players over time in sport. 00:24:10.228 --> 00:24:16.068 We modeled that there was a region in the middle of your career where 00:24:16.068 --> 00:24:17.978 that's what we're trying to estimate. 00:24:18.448 --> 00:24:22.958 And for example, in, in home run hitting, by the way, the optimal year for home 00:24:22.958 --> 00:24:24.748 run hitting is twenty-nine years old. 00:24:25.088 --> 00:24:27.038 For batting average, it's twenty-seven. 00:24:27.418 --> 00:24:31.688 So there's an, a, a region around that, and then we model age. 00:24:31.898 --> 00:24:39.108 How do players age in a, a specific young and a specific 00:24:39.108 --> 00:24:41.348 old parameter for each player? 00:24:41.348 --> 00:24:45.858 So we model even that some players age differently in that. 00:24:46.208 --> 00:24:51.708 Tiger Woods in golf has aged differently than, say, a Ben Hogan or a Jack 00:24:51.708 --> 00:24:57.748 Nicklaus in that, and he's aged poorly actually for, for well-known reasons. 00:24:58.208 --> 00:24:59.598 He- mostly health reasons. 00:25:00.628 --> 00:25:07.308 So we can model the age of those players, and when I talk about estimating their 00:25:07.308 --> 00:25:12.688 skill, I'm gonna talk about during that time of, of optimal performance. 00:25:12.948 --> 00:25:14.908 Now, there are a few players that are a little weird. 00:25:15.078 --> 00:25:19.098 Barry Bonds, for example, hit a large number of home runs 00:25:19.098 --> 00:25:20.278 when he was a little bit older. 00:25:20.718 --> 00:25:26.138 What it essentially sa- uh, estimates is that he aged incredibly well from, 00:25:26.168 --> 00:25:28.658 fro- over this time period, uh, in it. 00:25:29.548 --> 00:25:34.628 Now, we also model, important part 'cause I'm gonna come back to the populations. 00:25:34.968 --> 00:25:39.208 How good were players in the '20s, '30s, '40s, is we have a 00:25:39.208 --> 00:25:43.548 random effects model for players from the era in which they come. 00:25:43.948 --> 00:25:47.298 We actually did ten-year chunks in that. 00:25:47.298 --> 00:25:50.138 I would do it differently now, thirty years later, hopefully. 00:25:50.138 --> 00:25:55.468 Uh, hopefully as a statistician, I've aged better, and I would model this better now. 00:25:55.858 --> 00:26:00.738 Uh, within it, I would model it really as this sort of, uh, uh, moving 00:26:00.968 --> 00:26:05.338 normal dynamic linear model where there we did ten-year chunks in it. 00:26:05.598 --> 00:26:08.908 So a player comes from the era in which they were born. 00:26:09.928 --> 00:26:12.968 Now, I do wanna say one more thing about the data availability. 00:26:13.408 --> 00:26:19.268 When we were doing this in the late '90s, nowhere near the data availability now. 00:26:19.868 --> 00:26:26.428 And, um, my, my wife, my, my wife Tammy will remember, we bought a 00:26:26.428 --> 00:26:32.848 book called Total Hockey And we sat in a room where she read me 00:26:32.848 --> 00:26:37.158 the stats of NHL players over time. 00:26:37.628 --> 00:26:40.318 We drew a line and said you had to play a certain number of games. 00:26:40.318 --> 00:26:47.198 So she went page by page, she read me the stats and their birth date, and I 00:26:47.268 --> 00:26:54.268 typed them in, and we must have spent a month, multiple hours doing this. 00:26:54.808 --> 00:26:57.868 Pat Larkey was able to get it-- the, the data on golf. 00:26:57.868 --> 00:27:03.158 We ended up using only majors for golf because we just didn't have the data. 00:27:03.158 --> 00:27:07.028 Now you can go online, and I think you could, you could ask one of 00:27:07.028 --> 00:27:10.508 these large language models to get you this data and the birth date, 00:27:10.508 --> 00:27:12.028 and you have it in a, a minute. 00:27:12.618 --> 00:27:14.288 You'd have phenomenal data. 00:27:14.288 --> 00:27:18.098 In baseball as well, there's incredible resources for base-baseball. 00:27:19.078 --> 00:27:19.728 Uh, I... 00:27:19.728 --> 00:27:23.498 There were good data for baseball at the time, uh, wi-with ages. 00:27:23.498 --> 00:27:25.408 Hockey and golf was much more challenging. 00:27:26.398 --> 00:27:29.658 So the data availability, we, we didn't necessarily have it. 00:27:29.658 --> 00:27:30.438 We didn't actually... 00:27:30.658 --> 00:27:35.298 We couldn't find every golfer and, uh, in that, but much better data now. 00:27:36.578 --> 00:27:37.108 Okay. 00:27:37.848 --> 00:27:41.358 So just some results. 00:27:41.878 --> 00:27:43.378 And interestingly, at the... 00:27:43.378 --> 00:27:48.148 Remember we did this in the late nineties, and the best home run 00:27:48.148 --> 00:27:52.998 hitter, if you could pull everybody out at their elite, was Mark McGwire. 00:27:54.148 --> 00:27:58.388 It said Mark McGwire is the best home run hitter of all time, at that time. 00:27:58.568 --> 00:28:02.308 Tony Gwynn was the best batting average of all time, at that time. 00:28:04.038 --> 00:28:08.788 The best NHL scorer, rather surprisingly, was Mario Lemieux. 00:28:09.128 --> 00:28:12.118 It was not Wayne Gretzky, and I was shocked by that. 00:28:12.908 --> 00:28:13.978 And now I won't go in... 00:28:13.978 --> 00:28:17.678 I, I don't wanna spend too much detail on that, but largely, Wayne 00:28:17.678 --> 00:28:22.198 Gretzky played in an era where there was significantly more scoring. 00:28:22.578 --> 00:28:24.298 Now, he contributed to that. 00:28:24.798 --> 00:28:31.458 Uh, and Mario Lemieux, they did overlap in time within that, and Mario Lemieux, much 00:28:31.458 --> 00:28:36.118 of his career was a time where scoring was, was significantly lower, and yet 00:28:36.118 --> 00:28:39.338 he still posted, uh, crazy good numbers. 00:28:39.738 --> 00:28:42.288 And the best golfer of all time was Jack Nicklaus. 00:28:43.778 --> 00:28:48.458 Interestingly, that was nineteen ninety-seven, and in the dataset 00:28:48.458 --> 00:28:50.648 was this youngster, Tiger Woods. 00:28:50.648 --> 00:28:55.148 He was in the dataset, and he had won the nineteen ninety-seven 00:28:55.148 --> 00:29:00.438 Masters, famously going away, and he was very young at the time. 00:29:00.518 --> 00:29:02.618 I don't know, twenty, twenty-one years old maybe. 00:29:03.188 --> 00:29:08.198 And, um, and so that was really the only data in there, and the 00:29:08.198 --> 00:29:11.948 random effects model shrunk him, and he was number twenty on there. 00:29:12.558 --> 00:29:16.198 Shane Reese says, "Rerun this," and says Tiger Woods comes 00:29:16.198 --> 00:29:17.418 out as the best of all time. 00:29:18.448 --> 00:29:24.203 And, uh, I believe that And it, it's a similar thing to the Jesse Owens 00:29:24.203 --> 00:29:28.513 thing, the Johnny Weissmuller thing, for, for all of these, and I'll come 00:29:28.513 --> 00:29:30.463 back to the population part of it. 00:29:31.633 --> 00:29:37.513 Now, you can go in and look at the paper a- and look at these estimates. 00:29:37.513 --> 00:29:42.663 It also gives estimates of their career profile, and there's some 00:29:42.663 --> 00:29:47.273 certainly some interesting career profiles where some players, uh, aged 00:29:47.273 --> 00:29:51.153 much better, some aged worse, but this is a- at, at their peak time. 00:29:52.713 --> 00:29:57.143 And but I'm really interested in how performance changed over time. 00:29:57.143 --> 00:30:01.223 So I mentioned hockey and baseball are a little bit challenging, so when we 00:30:01.223 --> 00:30:07.213 look at how the population performs for home runs over time, it's a bit 00:30:07.213 --> 00:30:11.223 odd because there are some players in baseball who don't want to hit home runs. 00:30:11.983 --> 00:30:16.043 Ozzie Smith famously was an incredible defensive player, 00:30:16.583 --> 00:30:18.523 and he's in the Hall of Fame. 00:30:19.343 --> 00:30:20.783 He didn't hit home runs. 00:30:20.863 --> 00:30:23.443 You know, he might hit one or two a year accidentally. 00:30:23.893 --> 00:30:28.133 And so if you look at populations, how they behave over time, it's a bit 00:30:28.133 --> 00:30:32.023 misleading 'cause he's in there, and he's a quite poor home run hitter. 00:30:32.213 --> 00:30:34.803 So you don't wanna compare him to somebody at the other time. 00:30:35.273 --> 00:30:40.773 So hockey also, there are players in there who are not trying to score goals. 00:30:41.203 --> 00:30:43.223 Their, their job is not to score goals. 00:30:43.223 --> 00:30:44.043 They, they're, they... 00:30:44.043 --> 00:30:46.863 It's to or, or to assist on goals. 00:30:47.193 --> 00:30:49.903 It is to, uh, play defense. 00:30:50.523 --> 00:30:53.873 And so hockey is a little bit awkward as well. 00:30:54.303 --> 00:31:01.173 Now, the same pattern holds in both those sports despite this, is that 00:31:01.403 --> 00:31:06.333 we can look at the estimate of every player in this data set if they 00:31:06.333 --> 00:31:08.603 were to all play in the same season. 00:31:10.153 --> 00:31:15.673 That's the beauty of this time machine, is you can subtract time effects, and 00:31:15.673 --> 00:31:21.663 you can subtract age effects and say, you take at their peak, and they play 00:31:21.663 --> 00:31:23.723 at the same time, who's the best? 00:31:23.723 --> 00:31:28.073 That was the question I always was after here in that. 00:31:28.073 --> 00:31:33.283 So I wanna focus especially on golf because measuring golf is... 00:31:34.013 --> 00:31:36.413 There's only one goal in golf, to shoot a low number. 00:31:37.353 --> 00:31:41.053 So there is not this players, you know, are really good putters, 00:31:41.083 --> 00:31:43.583 but they shoot high numbers, but they're trying to be good putters. 00:31:43.983 --> 00:31:46.583 Uh, if you, if you did a scramble or something, it might be there, 00:31:46.583 --> 00:31:49.023 but golf is a pure thing at this. 00:31:49.673 --> 00:31:54.203 You see this incredible change when you look at the year they were born 00:31:54.603 --> 00:31:59.513 and their estimate of if they played all at the same time, you see this 00:31:59.953 --> 00:32:03.893 incredible downward Uh, to good scores. 00:32:04.433 --> 00:32:09.403 Downward progression of players over time in, in, in two ways you see, 00:32:09.643 --> 00:32:13.763 and we plot the median, the 90th percentile, and the 10th percentile. 00:32:13.803 --> 00:32:18.843 They all come down sharply from 1920 through... 00:32:18.843 --> 00:32:23.933 At this time it was sort of 19, yeah, you know, 65 through 1970 00:32:23.933 --> 00:32:26.973 were the years they were born by the time we did this analysis. 00:32:27.533 --> 00:32:29.173 There's a sharp decline. 00:32:29.903 --> 00:32:33.173 The ni- the 90th percentile, which is on the bad end of 00:32:33.173 --> 00:32:36.503 golf, decreases dramatically. 00:32:36.503 --> 00:32:39.573 So somebody born in 1930, they're... 00:32:41.103 --> 00:32:49.353 The, the 90th percentile was about 76 on this fictional, uh, same golf course. 00:32:49.763 --> 00:32:54.413 That 90th percentile for those born in the mid-1960s comes 00:32:54.413 --> 00:32:56.243 down to about 73 and a half. 00:32:56.733 --> 00:32:59.513 Two shots difference, which is enormous in golf. 00:33:00.373 --> 00:33:05.103 The median comes down, and that's about a shot plus different, 00:33:05.223 --> 00:33:06.413 which is also enormous. 00:33:07.193 --> 00:33:10.673 The 10th percentile goes down with a slightly less slope. 00:33:12.013 --> 00:33:18.033 So you see this huge shrinking of players down to the better ones reflective of 00:33:18.453 --> 00:33:24.073 there are many, many more players that could be on the PGA Tour and could play 00:33:24.073 --> 00:33:29.723 in these majors that weren't there in the 1930s, and it's reflective of this 00:33:29.723 --> 00:33:32.313 population all getting down to this. 00:33:32.313 --> 00:33:37.093 Now, some of the elite players at those times, as we talked about this 00:33:37.093 --> 00:33:38.743 extreme value theory, were there. 00:33:39.543 --> 00:33:43.923 And the Ben Hogans and the Sam Sneads, they were great players. 00:33:44.083 --> 00:33:49.773 Now, the median player at the time isn't nearly as good as the median 00:33:49.773 --> 00:33:55.183 player later within that, and this analysis is reflective of that. 00:33:55.183 --> 00:34:01.533 It says exactly that w- in this, in this time machine that 00:34:01.533 --> 00:34:03.003 we're able to look at this. 00:34:03.403 --> 00:34:06.953 An interesting thing about golf is my brother's in this data set. 00:34:07.903 --> 00:34:11.803 So, uh, my brother was born 1961, I think. 00:34:12.213 --> 00:34:19.853 Um, and he, um, he played in, uh, six majors, I think. 00:34:19.853 --> 00:34:23.613 He played in the 1991 US Open and he played in several PGAs. 00:34:23.613 --> 00:34:27.493 He's a professional golfer, and he was never on the PGA Tour. 00:34:27.493 --> 00:34:31.963 He went to Q-school, was a very, very good player at the time, but 00:34:31.963 --> 00:34:36.693 wasn't good enough to be on the PGA Tour, but was good enough to qualify 00:34:36.693 --> 00:34:38.633 for majors and played in majors. 00:34:39.103 --> 00:34:43.923 Interestingly, if you'd have taken him and moved him back 30 or 40 00:34:43.923 --> 00:34:49.343 years, he would have been better than the median player at those times. 00:34:50.153 --> 00:34:54.773 So it's, I, you know, it's, it's this time machine aspect, and we can measure this 00:34:54.773 --> 00:35:01.218 incredibly well in golf And by the way, golf is a sport that if we updated this, 00:35:01.218 --> 00:35:06.278 I think you'd even see drastically in the last, from the time we did this, in the 00:35:06.278 --> 00:35:12.588 last 30 years, because so many players now have access to play golf and to be 00:35:12.588 --> 00:35:15.248 in this data set and to be in the elite. 00:35:15.798 --> 00:35:17.718 And golf has exploded. 00:35:17.718 --> 00:35:18.908 Part of it is money. 00:35:19.278 --> 00:35:22.808 The money availability, the training, and all of that. 00:35:22.988 --> 00:35:24.948 Yeah, I know that, uh, uh... 00:35:24.978 --> 00:35:29.368 You know, I hit the ball farther now than I did when I was younger because 00:35:29.368 --> 00:35:34.278 of the equipment, but this is taking the equipment out of it because the, 00:35:34.278 --> 00:35:37.648 the round and the time pulls that out. 00:35:39.598 --> 00:35:40.318 Okay. 00:35:40.598 --> 00:35:46.468 So the population part of that, uh, comes out of that. 00:35:46.468 --> 00:35:49.508 Interestingly, when these results came out, it got quite a bit of 00:35:49.508 --> 00:35:53.848 press and people interested in that, and I, I wanna reflect on this. 00:35:53.848 --> 00:35:58.548 The LPGA Tour actually contacted me, and they were interested. 00:35:58.548 --> 00:36:01.548 I didn't run women's golf, and I felt bad all of a sudden when they 00:36:01.548 --> 00:36:05.648 contacted me, but they, they've always tried to figure out automatic 00:36:05.648 --> 00:36:08.218 qualifications for the Hall of Fame. 00:36:09.458 --> 00:36:14.128 And they were interested in whether a model like that could do that 00:36:14.128 --> 00:36:18.248 for the Hall of Fame, and it comes back to what I mentioned on this. 00:36:18.248 --> 00:36:20.068 This is not about Hall of Fame. 00:36:20.618 --> 00:36:26.338 It could say, for example, that all the women that played in the 1960s couldn't 00:36:26.338 --> 00:36:28.978 even play on the LB- PGA Tour right now. 00:36:29.248 --> 00:36:31.118 Doesn't mean they shouldn't be in the Hall of Fame. 00:36:31.118 --> 00:36:35.858 The Hall of Fame should measure relative to your peers, how did you perform? 00:36:35.888 --> 00:36:38.428 What did you accomplish in golf? 00:36:38.998 --> 00:36:42.408 Otherwise, we have to throw everybody out of the Hall of Fame, 00:36:42.408 --> 00:36:43.878 and it's only recent players. 00:36:43.928 --> 00:36:47.888 If it's purely objectively, how would that player have done? 00:36:49.208 --> 00:36:54.598 Now, the best of the best are much more comparable within that. 00:36:54.598 --> 00:36:58.288 By the way, Babe Ruth came out as the third-best home run hitter at that time. 00:36:59.038 --> 00:37:03.448 I think if you move that forward now, he probably drops into the teens or 00:37:03.448 --> 00:37:08.108 the 20s, that the world of, of home runs has changed dramatically in that. 00:37:08.708 --> 00:37:11.588 Uh, but Jack Nicklaus is comparable. 00:37:11.588 --> 00:37:18.878 You could take Jack Nicklaus of 1965 and move him now and playing, and 00:37:18.878 --> 00:37:20.968 he would be one of the best players. 00:37:23.498 --> 00:37:28.238 Canada, of course, cared about hockey and actually did a, a, a call-in show, 00:37:28.328 --> 00:37:32.428 uh, from a show in Winnipeg, and the... 00:37:32.458 --> 00:37:33.318 A call in... 00:37:33.318 --> 00:37:34.048 They, they did. 00:37:34.048 --> 00:37:38.068 They, they, they allowed listeners to call in and ask questions of me. 00:37:38.668 --> 00:37:42.738 And, and interestingly, they weren't all that worried about the Lemieux-Gretzky 00:37:42.738 --> 00:37:47.678 thing because they're both Canadian, and had one of them been a, a US, I think 00:37:47.818 --> 00:37:50.578 that, that they would've been upset about that, but they were both Canadian. 00:37:51.078 --> 00:37:54.883 But the, the caller said, "What does a Texan know about hockey?" 00:37:55.673 --> 00:37:57.933 And I said, "By the way, I grew up in Minneapolis." 00:37:57.933 --> 00:38:01.033 I actually grew up playing hockey, um, in, in that. 00:38:01.033 --> 00:38:03.423 I grew up in Minneapolis and immediately, "Oh, okay. 00:38:03.423 --> 00:38:04.183 That's okay then." 00:38:04.403 --> 00:38:08.143 It was so-- So it was sort of didn't matter my statistical chops. 00:38:08.143 --> 00:38:10.163 It's where do I know something about hockey? 00:38:10.473 --> 00:38:13.743 And, and then another caller came in and said, "Yeah, but your model 00:38:13.773 --> 00:38:18.743 can't measure heart, the heart of these players that played back then." 00:38:18.743 --> 00:38:22.553 Um, of course, my comment was, "Well, they should have scored more. 00:38:22.553 --> 00:38:25.343 They should have used heart to score more, uh, within that." 00:38:25.413 --> 00:38:29.183 But hockey again is, is much more complicated to model because, 00:38:29.583 --> 00:38:31.373 uh, of the differing goals. 00:38:32.033 --> 00:38:37.103 The UK cared about the golf and the US cared about home runs, mostly home runs. 00:38:37.103 --> 00:38:38.793 They didn't care so much about batting average. 00:38:40.413 --> 00:38:40.903 All right. 00:38:41.613 --> 00:38:46.783 So this has now been some 30 plus minutes. 00:38:47.093 --> 00:38:52.353 I haven't said a thing about clinical trials, and I did this work before 00:38:52.353 --> 00:38:54.443 I ever worked on a clinical trial. 00:38:54.863 --> 00:38:58.673 This was-- I was on faculty at Texas A&M, and I did this work 00:38:58.673 --> 00:39:01.563 and, um, I was fascinated by this. 00:39:01.563 --> 00:39:04.863 I was fascinated by the dynamics of populations in sports. 00:39:04.863 --> 00:39:06.323 Still am fascinated by this. 00:39:07.433 --> 00:39:10.963 And then I went into designing clinical trials. 00:39:10.963 --> 00:39:15.983 In 2000, we started Berry Consultants and started working on this, and soon 00:39:15.983 --> 00:39:19.543 after, uh, the following issue came up. 00:39:21.173 --> 00:39:27.563 We were involved, and Don Berry was the, the co-PI of the I-SPY 2 trial. 00:39:28.033 --> 00:39:34.523 It was a trial in, in neoadjuvant breast cancer where multiple arms would 00:39:34.523 --> 00:39:36.313 go into the trial at the same time. 00:39:37.733 --> 00:39:43.453 There's a control arm in the trial, and then arms A, B, and C come in, and 00:39:43.453 --> 00:39:47.663 it's a phase II trial, so they had as many as 120 patients, and they could 00:39:47.663 --> 00:39:49.823 adaptively stop this before that. 00:39:50.233 --> 00:39:53.373 And then that arm leaves the trial, but new arms come in. 00:39:53.673 --> 00:39:57.463 But there was a constant control over time, and then there are multiple arms. 00:39:57.463 --> 00:40:01.673 And it ended up, I believe, 27 or 28 different arms came 00:40:01.673 --> 00:40:03.773 in this trial over time. 00:40:05.063 --> 00:40:08.283 And at one point, they had to change the control 00:40:10.303 --> 00:40:16.073 because there was-- there were new treatments for that subgroup, 00:40:16.163 --> 00:40:18.503 that subtype of breast cancer. 00:40:18.563 --> 00:40:22.863 They're stratified by H- HER2 status and hormone receptor status, so it 00:40:22.863 --> 00:40:24.863 had a basket aspect to the trial. 00:40:24.893 --> 00:40:28.393 But within one of these baskets, there was a new control arm. 00:40:30.823 --> 00:40:35.423 The beauty of that was that control was actually in the trial as an 00:40:35.573 --> 00:40:40.463 e-experimental arm before that, and we w- we're gonna use that as a new 00:40:40.463 --> 00:40:46.633 control, but we're also interested in the new arms compared to the old control. 00:40:48.873 --> 00:40:56.833 So now we've got arm 21 in the trial and we're trying to compare it to an arm that 00:40:56.833 --> 00:40:59.373 was earlier, and they never overlapped. 00:41:00.003 --> 00:41:03.223 They didn't play in the trial in the same era. 00:41:04.573 --> 00:41:08.553 And the question was, how do we do these analyses for this new arm because 00:41:08.553 --> 00:41:13.343 we want to make that comparison, and we want to make a comparison to the 00:41:13.343 --> 00:41:17.113 new control, which by the way, was in earlier and then came in later, and 00:41:17.113 --> 00:41:19.393 it had a gap in time between that. 00:41:20.443 --> 00:41:22.613 This is exactly the same problem. 00:41:23.883 --> 00:41:28.343 So platform trials have exactly the same thing, where we're trying to 00:41:28.343 --> 00:41:31.963 make comparisons, direct comparisons. 00:41:33.043 --> 00:41:38.433 We don't wanna look at the raw data of what an arm did early in this trial to 00:41:38.433 --> 00:41:40.243 an-- what an arm did late in the trial. 00:41:40.433 --> 00:41:42.873 It's the Babe Ruth, Mark McGwire problem. 00:41:43.513 --> 00:41:46.073 It's different eras, but you don't just throw your arms up 00:41:46.073 --> 00:41:46.903 and say, "We can't do that." 00:41:46.903 --> 00:41:48.523 We can do that incredibly well. 00:41:49.553 --> 00:41:55.823 So we instituted the same model that was in that bridging eras of different 00:41:55.823 --> 00:41:58.313 sports into the I-SPY 2 trial. 00:41:59.193 --> 00:42:03.103 We institute this model in many, many platform trials because it's 00:42:03.103 --> 00:42:09.063 exactly this scenario in a-- this new thing called a platform trial 00:42:09.393 --> 00:42:12.323 as it is in that sports application. 00:42:13.323 --> 00:42:17.863 And we can pull two arms out, and we can make that direct comparison 00:42:18.243 --> 00:42:20.813 between those two arms by bridging. 00:42:21.303 --> 00:42:24.543 Now, what you want is to have this overlap. 00:42:24.543 --> 00:42:30.323 So arm three and arm fifteen weren't in at the same time, but arm three 00:42:30.323 --> 00:42:35.913 overlapped with four, um, overlapped with eight, overlapped with eleven, 00:42:35.913 --> 00:42:39.403 that overlapped with fifteen, and the other arms in there overlap. 00:42:39.403 --> 00:42:45.023 We can estimate time and what time does in the trial, the same thing 00:42:45.023 --> 00:42:47.083 we-- as we can estimate in sports. 00:42:51.853 --> 00:42:58.573 Now, we, we, we have a, a paper that talks about the Bayesian time 00:42:58.573 --> 00:43:01.573 machine, and it does e-exactly this. 00:43:01.853 --> 00:43:04.523 Saville, Berry, Berry, Veli and Berry. 00:43:04.523 --> 00:43:07.443 So you can look that up if that's of interest in there. 00:43:07.983 --> 00:43:09.953 But this exact same modeling. 00:43:11.053 --> 00:43:14.853 Now, let's go back for a second and think about what this means. 00:43:14.963 --> 00:43:16.773 Let's go back to the sports. 00:43:17.233 --> 00:43:20.113 Where does this model maybe break down? 00:43:22.263 --> 00:43:30.383 So if things are these additive effects of players, the model's perfect. 00:43:31.933 --> 00:43:37.703 So in sports, if there's an interaction, that's when it breaks down. 00:43:37.743 --> 00:43:40.163 Now, what is an interaction in sports mean? 00:43:41.288 --> 00:43:48.688 So if we're comparing Jack Nicklaus and his rounds in the 1960's to Scottie 00:43:48.688 --> 00:43:51.538 Scheffler's rounds in the twenty, uh, 2026 00:43:53.608 --> 00:43:56.758 the equipment's very, very different, no question about it. 00:43:56.758 --> 00:43:58.878 Golf balls are very, very different. 00:44:00.148 --> 00:44:03.598 But we're not, we're not saying Jack has to bring his equipment. 00:44:03.628 --> 00:44:07.898 He could play with today's equipment when he was nine-- in 1965. 00:44:09.628 --> 00:44:14.258 But suppose the new sand wedges and the new balls they have, 00:44:14.688 --> 00:44:19.308 that Jack just can't utilize them the way Scottie Scheffler can. 00:44:19.958 --> 00:44:25.248 And-- But yet you didn't have sand wedges of the same, uh, incredible... 00:44:25.488 --> 00:44:28.298 That's one thing that's actually changed over time with a golf ball. 00:44:29.008 --> 00:44:34.308 That there's this interaction between them, and, um, moving Scottie 00:44:34.308 --> 00:44:38.478 Scheffler back would have not just an additive effect, but would have 00:44:38.478 --> 00:44:42.378 an interaction because he couldn't play with the wedges of the time. 00:44:43.568 --> 00:44:50.558 Now, I think these are strikingly rare i-in, in in sports that these 00:44:50.558 --> 00:44:56.208 interactions-- There may be a little bit to a dynamic sport like hockey, where a 00:44:56.208 --> 00:45:02.198 player in the 1940s was much smaller, and it may be harder to kind of do the same 00:45:02.198 --> 00:45:04.558 things they did where players are larger. 00:45:05.008 --> 00:45:10.228 But mostly, you gotta work really, really hard to figure 00:45:10.228 --> 00:45:12.538 out interactions in those sports. 00:45:12.538 --> 00:45:16.498 And I know you can come up with them, but, uh, I, I want you to think really 00:45:16.498 --> 00:45:18.898 hard about whether it, it, it's reality. 00:45:20.088 --> 00:45:23.868 Now, in clinical trials, it's the exact same thing. 00:45:24.018 --> 00:45:26.048 That's the potential challenge here. 00:45:26.778 --> 00:45:30.678 Is when we do this adjustment, are there interactions? 00:45:31.918 --> 00:45:37.408 I would say that they're actually less likely in clinical trials 00:45:37.758 --> 00:45:40.088 than they were in sports. 00:45:40.348 --> 00:45:43.278 By the way, interestingly, the, the beautiful thing about clinical trials 00:45:43.278 --> 00:45:44.428 is we don't have the age effect. 00:45:46.138 --> 00:45:50.238 You don't have to model how players age in this bridging 00:45:50.238 --> 00:45:52.628 part of it in clinical trials. 00:45:52.888 --> 00:45:55.128 Drugs don't age within that. 00:45:55.488 --> 00:45:57.608 Now, what might an interaction be? 00:45:57.978 --> 00:46:02.538 And the, the prime candidate for where there might be interactions are probably 00:46:02.538 --> 00:46:09.968 infectious disease, where an old trial had a result relative to placebo, 00:46:10.708 --> 00:46:12.768 and now the disease is different. 00:46:12.798 --> 00:46:18.988 So the current trial, maybe that treatment wouldn't be as good Even 00:46:18.988 --> 00:46:21.298 those I think are, are unlikely. 00:46:21.748 --> 00:46:26.098 COVID was the one prime example because the disease morphed so 00:46:26.158 --> 00:46:27.758 quickly and changed so much. 00:46:28.058 --> 00:46:32.388 Now, most of the hospitalized trials we did in COVID, it's about the host, and 00:46:32.388 --> 00:46:37.188 the host is sick, and it's not really about the, the, the viral-- virus anymore. 00:46:37.188 --> 00:46:41.888 But even in that case, it's not clear that there are interactions. 00:46:42.798 --> 00:46:47.508 And by the way, if we think there are interactions where treatment was in a 00:46:47.508 --> 00:46:54.388 trial before an overlap placebo, and now we have this overlap of arms, and 00:46:54.388 --> 00:47:01.788 we think that the relative differences of the arms could switch, we can't 00:47:01.788 --> 00:47:04.288 do non-inferiority trials anymore. 00:47:04.848 --> 00:47:08.618 And actually, when a trial's over, I'm-- I don't even know if we can think does the 00:47:08.618 --> 00:47:13.838 drug work because right now it might have switched with placebo, uh, within that, 00:47:14.188 --> 00:47:16.208 and do we know if the treatment works? 00:47:16.438 --> 00:47:17.498 So if you're looking at... 00:47:17.638 --> 00:47:23.308 And, and what's really interesting, we can test this because of an arm's length of 00:47:23.308 --> 00:47:28.148 its career in a trial and the different arms, we can look to see whether it 00:47:28.408 --> 00:47:31.018 looks like it varies relative to control. 00:47:31.678 --> 00:47:36.338 In I-SPY 2, if you took the same treatment in there at different 00:47:36.338 --> 00:47:38.698 times, it was incredibly stable. 00:47:39.718 --> 00:47:45.788 Neoadjuvant breast cancer, whether this is ALS, whether this is, um, a 00:47:45.788 --> 00:47:52.438 cardiovascular disease, weight loss, this is, um, uh, many other diseases, 00:47:52.998 --> 00:47:56.648 it's really hard to think about interactions within those trials. 00:47:56.648 --> 00:48:00.208 But that would be the same way in which in a platform trial you would 00:48:00.208 --> 00:48:09.208 be concerned about using this model to compare treatments that are in the trial 00:48:09.268 --> 00:48:11.598 at different times and different eras. 00:48:14.468 --> 00:48:15.508 Okay. 00:48:15.858 --> 00:48:16.458 Well, 00:48:19.028 --> 00:48:22.238 uh, by the way, if you're, if you're interested in this, you can jump 00:48:22.238 --> 00:48:27.108 to episodes twenty and twenty-one, talk about I-SPY 2 with, with Don. 00:48:27.458 --> 00:48:31.718 Um, a incredible story in and of itself, but part of that 00:48:31.718 --> 00:48:33.138 talks about the time machine. 00:48:33.528 --> 00:48:36.998 And then episode twenty-two talks about the time machine much more 00:48:36.998 --> 00:48:40.828 from a statistical standpoint with, uh, co-host Kurt Vile. 00:48:41.258 --> 00:48:42.988 So those may be of interest to you. 00:48:43.618 --> 00:48:49.748 So coming back to the original lead-in to this, uh, Cooper's 00:48:49.748 --> 00:48:52.608 team against the 1927 Yankees. 00:48:52.638 --> 00:48:56.218 By the way, we don't have the data to make that bridging because 00:48:56.218 --> 00:49:01.028 we don't have Division III teams playing pro teams and even Division 00:49:01.028 --> 00:49:03.308 I teams playing pro teams in that. 00:49:03.908 --> 00:49:09.658 Uh, though, though my son's team did scrimmage, uh, this year a, um, 00:49:10.118 --> 00:49:12.478 a rookie league professional team. 00:49:13.178 --> 00:49:19.218 Um, in college hockey I think it's clear a college hockey team 00:49:19.218 --> 00:49:22.248 now would beat a 1920s NHL team. 00:49:23.028 --> 00:49:27.768 E- And yes, they would use the same equipment, so the, that NHL team could 00:49:27.768 --> 00:49:30.738 get a little bit of time to practice and use the equipment and all of that. 00:49:30.738 --> 00:49:34.938 So it, it, you know, seamlessly that, I, I think a college hockey team now 00:49:34.938 --> 00:49:37.228 would beat a team back then for sure. 00:49:37.678 --> 00:49:43.108 I think an NCAA Division I baseball team now beats the 1927 Yankees. 00:49:43.728 --> 00:49:48.398 I think it, it's just the world has moved on, the, the speed of these pitchers. 00:49:48.398 --> 00:49:51.418 I don't know if any 1927 Yankees threw 90 miles an hour. 00:49:51.418 --> 00:49:52.188 I don't think they did. 00:49:52.818 --> 00:49:56.738 I think a D1 team would, would beat up the 1927 Yankees. 00:49:57.018 --> 00:49:58.668 And in golf, it's absolutely clear. 00:49:58.838 --> 00:50:03.688 NCAA golfers now are better than the, the, the 1920s. 00:50:04.178 --> 00:50:07.518 I, I know there's Bobby Jones and other players like that, but 00:50:07.548 --> 00:50:12.868 they would, I think they would, quote-unquote, "beat up" on 1927 golfers. 00:50:13.818 --> 00:50:19.048 So where does that leave Division III baseball team now against the '27 Yankees? 00:50:19.048 --> 00:50:20.658 I don't know the answer to that. 00:50:20.658 --> 00:50:22.848 I think it would actually be a really good game. 00:50:23.828 --> 00:50:27.708 And maybe Babe Ruth calls his shot and hits the winning home 00:50:27.708 --> 00:50:30.028 run, uh, in a game like that. 00:50:30.058 --> 00:50:33.308 But I actually think it would be a reasonable competition. 00:50:35.527 --> 00:50:36.547 All right. 00:50:36.547 --> 00:50:41.457 Well, um, this is coming to you through a time machine, by the way. 00:50:41.817 --> 00:50:46.217 It's recorded, and through time you got to hear this through a time 00:50:46.217 --> 00:50:48.197 machine about The Time Machine. 00:50:49.477 --> 00:50:59.137 And next time, the next episode may be better because of, of, uh, era effects. 00:50:59.477 --> 00:51:03.637 But until the next time, we'll be here in the interim