Mammograms: Death Threats, Hillary Clinton and Lead-Time Bias

In this episode of "In the Interim…", Dr. Don Berry provides a detailed account of co-chairing the 1997 NIH consensus development conference on mammography for women in their 40s. His conversation with Dr. Scott Berry covers the statistical and clinical complexities of breast cancer screening, addressing lead time and length bias, trial design limitations. Don discusses the panel’s finding, based on randomized trials and meta-analysis, that the average benefit of screening women in their 40s is modest (an estimated 1.4-day average life extension and 18% hazard reduction). The panel recommended individualized decision-making rather than universal screening. The episode follows the reaction: heated debate with radiologists, scrutiny from journalists and policymakers, Senate testimony, and personal threats. Don explains how these findings became distilled into soundbites, evidenced by coverage in The New York Times, Chicago Tribune, and a reference in James Patterson’s Murder Games. The conversation addresses overdiagnosis, false positives, and the consequence some called the “Berry effect”—an observed drop in mammogram rates after the panel’s recommendations.

Key Highlights

• Don Berry’s NIH consensus conference leadership and approach to breast cancer screening recommendations.
• Statistical bias and trial limitations inherent in mammography evidence.
• Meta-analysis findings: 18% hazard reduction; 1.4-day average life extension.
• Policy guidance supporting individualized patient decision-making over universal screening.
• Strong backlash from advocacy, media, radiology, and government—including Senate hearings and personal threats.
• Enduring debates, public misunderstandings, and continued citation of these events in scientific and popular sources.

For more, visit us at https://www.berryconsultants.com/

Creators and Guests

Host

Scott Berry

President and a Senior Statistical Scientist at Berry Consultants, LLC

What is In the Interim...?

A podcast on statistical science and clinical trials.

Explore the intricacies of Bayesian statistics and adaptive clinical trials. Uncover methods that push beyond conventional paradigms, ushering in data-driven insights that enhance trial outcomes while ensuring safety and efficacy. Join us as we dive into complex medical challenges and regulatory landscapes, offering innovative solutions tailored for pharma pioneers. Featuring expertise from industry leaders, each episode is crafted to provide clarity, foster debate, and challenge mainstream perspectives, ensuring you remain at the forefront of clinical trial excellence.

Judith: Welcome to Berry's In the
Interim podcast, where we explore the

cutting edge of innovative clinical
trial design for the pharmaceutical and

medical industries, and so much more.

Let's dive in.

Scott: Welcome everybody
back to In the Interim.

I'm your host, Scott Berry.

And today, uh, on In the Interim,
we are going to go back in time.

We're gonna go back 30 years.

We'll probably go a little bit
further to set up the story as well.

And really, an amazing story of science
and the impact of statisticians and

the controversy, uh, of this impact.

So joining me today is the
author of this impact, Dr.

Don Berry, and yes, he's my father,
and a frequent guest on In the Interim.

So welcome back to In the Interim.

Donald Berry: Thank you, Scott.

I hope this goes well, and we
don't create any more controversy

Scott: That's right.

That's right.

So I, I, I guess we wanna go
back to set up this story.

We wanna go back to perhaps maybe
even 1990, and you, um, a- agreeing

to join the Cancer and Leukemia
Group B, CALGB, uh, committee.

You, you, you move on to Duke,
and now you're getting into

oncology, uh, uh, statistics

Donald Berry: Yes.

Uh, I was the,

uh, faculty statistician on
a committee for the CLGB, the

Cancer and Leukemia Group B.

Uh, I was a faculty statistician
on the breast committee.

And so our goal was-- And we had,
you know, NIH, uh, NCI funding

to do clinical trials, um, and,
uh, with, uh, with great impact.

And it's-- It was sort of the proving
grounds for will MDs accept these, uh,

lunatic things I've been talking about?

And it was, um…

By the way, that's chronicled in, uh, the,
the article that we, we might mention, the

one that was, uh, in Science magazine, uh,
by, uh, uh, a-authored by Jennifer Cousin.

And, uh, you came down, Scott, uh,
to meet with her when she visited,

uh, uh, Houston and MD Anderson
and me to discuss, you know, how…

Uh, I think the title of the paper was,
uh, uh, something like the new math.

What-- How the new math is, uh, is

Uh, changing things.

And of course, we, we
continue to change things

Scott: So you start, you start as a lead
statistician, so you're designing trials,

and, and this in and of itself is an
interesting story of, you know, letting

the Bayesian in, the-- what people thought
of you as being loosey-goosey, and, uh,

then coming around to Bayesian design.

And I guess that leads up to you
being invited by the NIH to co-chair

a consensus panel, uh, for them,
and we'll describe the topic.

Do you remember this, h-how this
came about, where you're invited

to co-chair this consensus panel?

Donald Berry: Uh, yeah, I just
wanna correct one thing and,

and it might, uh, be confusing.

Uh, when I was at the CLGB, uh, you're
right, eventually we came to Bayesian, but

it wasn't-- uh, f- it was far from an easy
thing, and I th- I became a frequentist.

I mean, it was the only
way I could get by.

So it was like, you know, the, the, the,
the bullfighter and the, uh, clown, uh,

that, uh, you know, attracts the bull.

Uh, that was me for a while.

But eventually, they listened
to the bull that I was spewing

and, uh, the, you know, the…

When I, when I started out, it was,
"What does the FDA think about this?"

And when it ended up, it was,
"What does Don think about this?"

So it was, uh, a-a-and that was, uh,
you know, uh, at least a decade in total

So

Scott: So you start this in, in 1990, you
start this, and so you, you, you, you play

this role of, of standard statistician.

You're not rocking the
boat and all of that.

But by, by say 1997, when you're asked
to do this consensus panel, there's a

great deal of trust and, uh, understanding
in your knowledge in breast cancer.

This is a consensus panel on whether women
in their 40s should have a mammogram,

and so you're asked to co-chair this

Donald Berry: Yeah, and, and it, it--
there are-- I don't know whether there,

there still are, but at the time,
there were these consensus development

conferences, that's the official
name, for various things in medicine.

And anything that was, uh, important
in medicine qualified as being one of

the consensus development conferences.

So, for example, they had one in, um,
uh, DCIS, ductal carcinoma in situ.

Is it a disease?

I mean, is it a cancer?

Um, they had one, uh…

I mean, the only other one that I was
on besides this, uh, the bre- the,

uh, women in their 40s, uh, was, uh,
ADHD, uh, and Ritalin, uh, and its use.

And, uh, I mention that
because it was not a consensus.

I was the odd person out.

Uh, there were, uh, MDs that were
doing this, and they were on one

side, I was on the other side.

The only consensus we had
was, uh, there was a problem.

Uh, you know, the use of Ritalin some
places was over 50% in individual

classrooms, and, um, it is hard
to imagine that there are that

many, uh, uh, bad things going on.

Anyway, this was a consensus.

We did, did develop a consensus.

Uh, the chair, uh, was Leon
Gordis, who was an epidemiologist

at, um, uh, Johns Hopkins.

Uh, there were radiologists.

And the eventual, uh, uh, uh, document
that we produced and the eventual

announcement that we, uh, made at
this conference was a two-day thing

in Washington and, and Bethesda.

And, um, the first day, there were
presentations from people who did-- had

done studies, uh, screening studies,
the PIs, uh, many from, uh, Sweden.

Sweden was a major player in this.

Uh, and so they came over
and presented their data.

There were some data from other places
as well, but Sweden was the, kind

of the, the, the domineering thing.

And there really was a consensus,
and they made us do that because they

kept us in the room until, like, 3:00
plus in the morning of the second day.

So you didn't get much sleep, but
we, we got a, a good bit done.

Scott: So the consensus statement
that comes out of that is that

the panel concludes that the data
currently available do not warrant

a universal recommendation for
mammography all women in their forties.

woman should decide for herself
whether to undergo mammography

Donald Berry: Correct.

And the paper that we published and
the announcement, um, but, uh, uh,

you know, that sounds pretty good.

Uh, I mean, it sounds like…

Scott: you.

Donald Berry: I said, "Oh, it put it…

W- w- this may be appropriate for some
women, and they should discuss and should

learn, you know, what is the benefit?

What are, what are the harms?"

Uh, by the way, the harms are clear.

The benefits are not so clear, and
that's what w- the major aspect of this

conference, development conference was,
was what is the, um, the life extension?

I mean, do, do, do we make
women, uh, uh, longer?

W- do we prolong the lives of
women in their 40s if they start

taking, uh, mammograms either…

And, and w- we'll come across the
question of, uh, is it annual mammograms?

Is it biannual?

Is it triannual?

Um, so…

And it was a k- kind
of an eerie experience.

It was k- like a religious, uh, revival
meeting where when something good was

said from the podium, there was, uh, uh,
olés and, and, and, uh, an, an applause.

When something, uh, uh, bad,
uh, there was, uh, there were no

tomatoes there, but it was cafás.

So it was, uh…

A- and, and as the presentations
were made, there were people doing

calculations in the audience and,
uh, you know, they'd update their

meta-analysis, uh, for the-- based
on the r- the, uh, uh, the randomized

screening trials, um, and ad- addressing
the quality of the trials and the like.

Uh, and it was, it was scary.

And on the second day, um, they--
we presented, uh, Leon and I

presented the, uh, re- the Outcome.

What was the consensus?

What was-- what are the, uh,
negatives of, uh, screening?

What are the positives of screening?

How, how well do we know them?

Um, and, uh, then Rick Klausner, uh, came
up and he-- Rick Klausner was the head of

the Ca-- National Cancer Institute at the
time, the, the sponsor of this shindig.

Um, and he said, essentially,
"Pay no attention to these people.

You've got to screen,
uh, women in their 40s."

And it was-- he was
under enormous pressure.

Arlen Specter, one of the, uh,
senators at, at the time, uh, senator

from Pennsylvania, was chair of the
committee that it was appropriate

for the reporting of the NCI and the
providing the budget for the NCI.

And Specter had told Klausner
that, "You have to recommend

screening for women in their 40s."

And there was a, uh, uh, uh, uh, an, an
announcement made that the, uh, senators

had voted on this question and the
document, i-i-it's, it's kind of funny

Uh, it, it, a bit like the state
legislature that, uh, approved a bill

that said in that state, and I won't
tell you which state, in that state,

pi, P-I, the mathematical constant,
uh, is herewith, uh, going to be 3.14

because we don't want
this complicated stuff.

We want people to understand it.

It's, uh, like, uh, uh…

So what the, the statement said
was essentially that screening

will be appropriate and is
good for women in their 40s.

Uh, so it was, it was a really weird
experience and during the course of

this, uh, meeting, somewhere along the
way, the people doing the meta-analysis

is another, uh, trial was announced.

Uh, got up to the microphone and said,
"We now have statistical significance,"

and "Hip hip, yay, yay, yay, yay."

Uh, and those of you that are
statisticians know that that's a bit of

a problem to do these interim analyses
and, uh, you know, when you get a

positive result, you declare, uh, yay.

Uh, so it was, it was
scary in lots of ways

Scott: So you're, you're-- we'll get,
come to the backlash a little bit

and, um, the fallout of all of this.

So talk a little bit
about the science of this.

First, it seems like statistical
problems in mammography or trials are,

are-- have huge number of challenges.

The challenges in these trials
are, uh, of course, lead time bias.

Imagine doing a, a-- looking at data for
those that are diagnosed from a mammogram

as opposed to di- diagnosed in an
alternative way, in a non-mammogram way.

Those are different time points in the
course of a cancer, and then looking

at the behavior of those outcomes.

There's a great deal of
crossover in these trials.

So the, the science of these
trials and understanding are

mammograms, do they save lives?

They're, they're very m- a
number of challenging trials

Donald Berry: Yeah.

So the-- what you're talking
about, the trials that were, um,

uh, considered by this committee,
consensus development committee,

were only the randomized trials.

And the, uh, randomized trials
are not susceptible to this

lead time bias or length bias.

Um, uh, length bias is even more
important than lead time bias.

Length bias is when you, um, detect a
tumor, you detect the tumor that, uh,

is, uh, s- uh, otherwise not detectable
or doesn't, you know, symptoms haven't

been, uh, uh, uh, observed yet.

Um, and you tend to detect the
tumors, the incremental tumors

that are-- have a, a long sojourn
time, that is, uh, slowly growing.

And if they're slowly growing,
they tend to be, um, you know,

the, the, the patient lives longer.

And so if you do a-- if you don't do a
randomization, if you'd have a database

where you, uh, examine the patients and,
and follow the patients, uh, and then

you ask the question: How is it detected?

How is the tumor, the tumor detected?

If it's detected by screening

The patients live longer.

I don't mean, you know,
a couple of weeks longer.

I mean years longer
because of these biases.

Um, and so th- and there are…

A-a-and I don't exaggerate, there
are thousands of studies that do this

in various cancers that, that are,
that are published and you can…

And, and journals don't understand it.

I mean, some journals are, are getting
to understand, but, uh, a, an article in

radiology is not submitted to the strict
statistical, uh, uh, demands, and you

see some th- some stuff that's really
questionable there or in the journal

Cancer, um, uh, the, the same thing.

So what you, y-you're talking
about two things, Scott.

One is the trials.

The trials don't have this problem.

Um, the…

But the basis for the bias out
there, there are people now,

radiologists, uh, others in the medical
community who believe the result

because it was in their database.

They attribute the benefit to the
screening, and they publish things,

you know, with 70% reduction in the
risk of mortality if you're screened.

And it's like a religion, in
this case, a false religion

Scott: Sure.

Yeah.

So you did have, uh, during the
course of this, there were a number of

randomized trials where, where women
are randomized to screening in their 40s

or not screening in their 40s, and then
you're looking going forward in that.

This is not about method of detection.

So you, you had some reasonable things.

You were doing meta-analyses of these.

You published in JNCI, uh, a
meta-analysis of various studies.

So part of this then becomes, while
you're making this, is the, the

size of any potential benefit, the
uncertainty of is there benefit?

But you're also weighing, as
you said, the, the downsides of

mammograms, uh, safety of them,
uh, downsides of false positives.

This all goes into this eventual
recommendation that the committee makes.

Donald Berry: Uh, yeah, correct

So

What's the question?

Scott: That, that, that was the question.

That I'm assuming this played a
really important role is I think

you, you, you did have some level
of statistical significance for

benefit, uh, o-overall of the data.

But it was, for example, a five-day
benefit over the course of a lifetime

was one of the estimates that comes
out of JNCI as a really small effect.

But yet also there are the, the,
the safety aspects of this, the

false positives that happened
during this, and the ramifications

of false, false positives.

So I'm as- uh, I mean, that sounds like
that was an important part of the decision

Donald Berry: Yes.

Uh, eh, it was, uh,
rather more complicated.

The, uh, women in their 40s,
for example, was a subset.

In most of the trials, it was,
uh, women that are 40 to 74, uh,

that were included in the trials.

And so age had an effect.

Uh, the different trials had different,
uh, uh, periods of, of, uh, screening.

So some were, uh, one year
every year, annual screening.

Some were, uh, every two years.

Some were kind of a gemish of that.

Um, and, um, the, uh, the issue that,
uh, we probably will come to today,

the issue of so Uh, why screen?

You wanna find early.

Was it, is it finding early
or is it finding early and

giving treatment earlier?

Uh, I mean, if there's no, uh,
effect of anything that you do

after you find it, the tumor,
then there's no reason to do it.

Um, and is that finding surgery?

I mean, all of these women
would have the tumor removed.

Um, but is it, uh, importantly
the stuff that comes afterwards?

And over the last 30 years,
things have changed dramatically,

uh, as regards to that.

These randomized trials that we
looked at had, by and large, no

treatment except for surgery.

Um, and, uh, today, of course, uh, there
are, uh, you know, hundreds literally of

possible treatments that one can use and
there's endocrine therapies, tamoxifen

that's used for estrogen receptor-positive
patients that is hugely beneficial,

probably has saved more lives than any
other drug, um, uh, ever, although we've

seen some, uh, uh, amazing things since.

Um, and, uh, chemotherapy,
chemotherapy, uh, tamoxifen is

regarded to be endocrine therapy.

It's a kind of a, a, a warmer and
gentler thing than, uh, uh, chemotherapy,

which kills lots of cells, including
cells that you don't want it to kill.

So it's a complicated, it's a complicated
story, and we were asked to address

those things, addressing things like,
um, let's assume that screening is

effective Should it be used annually?

Should it be used biannually?

And, uh, the statisticians in the
crowd will know that discriminating

between those two things is--
it's kind of like a dose.

You know, a low dose and
a slightly higher dose.

Distinguishing efficacy between
those is really difficult.

Um, so it's a, it's a complicated story,
uh, uh, w- that had to have an answer.

What should the rec- the
REMAC recommendation?

And this thing, by the way, that Harlan
Specter said, "You don't get paid unless

you-- You don't get your budget unless
you, um, have, have a recommendation

that women in their 40s benefit from,
um, from screening," uh, that was

sort of wrongheaded from the get-go.

The NCI never makes a recommendation.

They have-- There are committees that
are out there that make recommendation,

but the NCI never makes a recommendation,
at least never until this.

Um, recently, somebody found this
recommendation somewhere in the NCI

bowels and, uh, uh, trashed it, you
know, got rid of it because it, it

really was the only one that they had.

Anyway, making the recommendations,
uh, we were, we were supposed to make a

recom- a recommendation, uh, and we did

Scott: Yeah.

So th-- and so after this, after
this recommendation, backlash starts.

there's a number of reports in
the media, and I think you said

that you've been quoted in over
100 media sources about screening.

Uh, but there's a great deal of media
on this, and negative, uh, generally.

Uh, some of it not quite as negative,

Donald Berry: Well,

Scott: claims of

Donald Berry: just to, to
correct that, the, there are some

journalists who are not duped.

So Gina Kolata from The New York Times,
um, at the time, uh, John Crewdson from a

Pulitzer Prize, uh, uh, a person from, uh,
the Chicago Tribune, uh, Judy Perez from

the Tribune, um, who are really into it.

And Judy Perez-- And they've written
editorials, uh, for their newspapers,

uh, that they just, uh, uh, uh,
lambasted things like the Journal

Cancer because it publishes this stuff.

Um, and, uh, so it's, it's a…

It, it's the journalists
are saying like it is.

So for example, uh, The New York Times
had a headline after this conference that

said, uh, that the panel got together
to, uh, present, uh, the, the ac- the

experts, uh, to present to the, um,
uh, NCI, uh, audience, which was huge.

Um, and then they recommended mammograms.

Uh, uh, the, the people who were
the players like, uh, Rick, uh,

Klausner, the, uh, NCI director.

So they were reporting This stuff,
and they used terms like the

mammogram wars, and the mammogram
wars were among the scientists.

Um, and explaining to, um, the
politicians, which is, as I think

everybody knows, um, they listen
to Will Rogers, who knew, um, uh,

that they are not very scientific.

So how do you convey this?

And that was the next chapter in this.

And for example…

I mean, there's a, there's a committee
going on today, um, uh, with, um,

the Senate investigating Tony Fauci.

It was this committee, you know, with
different people in the committee,

uh, that, uh, wanted a, uh, meeting
where the science would be discussed.

And

Scott: R- maybe let's wait till, let's
wait for the Senate, uh, for a minute.

But, but, uh, a- and 'cause that happens
a few years afterwards, actually,

Donald Berry: Yeah, uh

Scott: uh, in the meantime, you
are, uh, you get a great deal

of backlash from radiologists.

You do multiple duels where you're
discussing, and it's really, you

know, Don Berry against, uh, uh,
Copens and, and the views of this.

You receive death threats.

Um, you, you get accu- uh, the potentially
suing, uh, the committee suing you.

This gets, um, quite heated, uh, a
great deal of passion, uh, that your

committee got it wrong, um, in all of this

Donald Berry: Yes.

Uh, one thing to fill in
there, uh, so I went to Europe

and, uh, uh, at a conference.

Uh, by the way, I-- uh, Dan Kopans, the
person you mentioned, is a, uh, ardent

radiologist in this, uh, that, that, um,
uh, is somebody that, uh, is sort of the

spokesperson or was for radiologists.

Um, and, uh, and, and, and one year,
I forget which year it was, probably

like the year 2000, uh, uh, he and
I went toe-to-toe, head-to-head, uh,

in duels that were sponsored by, uh,
national, uh, um, uh, cooperative, uh,

uh, groups, for example, um, where it
was advertised as Berry versus Kopans.

Uh, I, I'm sorry, Dan, but I won all three

And, a-and, a-and the other, there

When I went to Europe, uh, lots of people
were there from Sweden for my talk.

And, um, when I came back, uh, one of
them had written a letter to people in the

United States, lots of politician, lots
of people from the, the, the Senate, uh,

lots of people from the administration.

Uh, so for example, the director of
the, uh, National Cancer Institute,

uh, uh, the, the president of MD
Anderson, my institution, um, saying…

Uh, it was, it, it was,
it, it was disgusting.

It was, um, ad hominem attacks,
uh, things I'll tell you that, uh,

you know, almost all were false.

I won't tell you any of them
because it's embarrassing.

In fact, I never quite finished the,
the letter because it was so disgusting.

My reaction, um…

I mean, these were clearly
personal things, things that

I was, I was a bad person.

Anybody would think somebody
like this is, is a bad person.

Um, and so I, I wrote a one-sentence note
and sent back and I said, "I wanna thank

you for not sending this to my mother."

Scott: Yeah.

Donald Berry: It was,

Scott: Yeah.

Donald Berry: uh, it really,
this wasn't Dan Kopans, by

the way, that was writing it

Scott: Yeah.

Yeah.

Yeah.

Okay, a- and, and part of the backdrop,
part of the, the aspect about mammograms

is there are, there are a lot of women out
there who have had a mammogram and have

found cancer, and to this day believe that
they're alive because of the mammogram.

And so you can imagine the huge amount…

You don't have to imagine it, you saw
it, the huge amount of passion and

belief that you, Don Berry, you have
no idea what you're talking about.

Look at all the people that this has saved

Donald Berry: Yeah, and they reverse
it too to say, "Look at all the people

you're killing," literally accused of
killing women, which was n- you know

Scott: So now, n- now the, the, the, the,
the legislature, politicians, uh, see the,

the, the, the uproar, the backlash, the,
as, as the articles say, the, the brawling

over mammography, the mammography war.

And so they wanna have a, a
fact-finding, so they bring you in to

a Senate panel to discuss this issue

Donald Berry: Yes

Scott: And so you, you
testify, um, uh, this.

And so tell us about your
experience testifying in

front of Senate on this issue

Donald Berry: So it was, um,
it was quite an experience.

Um, the senators were, um, on, on
this, uh, uh, committee was Bill Frist.

He was the, um, the Senate, uh, major-
uh, majority leader at the time.

Um, uh, Tom, uh, Harkin
was a, a senator from Iowa.

Um, the, um-

The people that were testifying
were the, of course, uh, Andy von

Eschenbach, the director of the NCI,
um, somebody from cancer, uh, the

American Cancer Society, um, and me.

When Bill Frist…

So what, what, you know, uh,
the, the, the regimen, uh,

the senator gets five minutes.

Uh, Bill Frist spent five minutes
saying the following over and over

But you're not a medical
doctor, are you, Dr.

Berry?

Uh, and I was, I was warned by, by the
staff of the Senate that, uh, f-- uh, Bill

Frist was the only MD in the Senate, and
he made sure that everybody knew that.

So it was, you know…

He, he never asked a substantive question.

Um, he just kept repeating
this thing, "But you're not

a medical doctor, are you?"

So, um, I, of course, uh, had to agree.

Um, and, um, following me, uh, uh, on
the, uh, uh, uh, at, at, at the mic, uh,

and presenting her, uh, attitude was Fran
Visco, who was the head of the, um, the,

the biggest network of breast cancer,
uh, patients, uh, advocacy, uh, group.

And so she, in her summary,
started out by saying, "Dr.

Frist, statisticians are the
experts on this question."

Scott: That's just brilliant.

That's brilliant.

Donald Berry: Uh, two, so two, two
other, two other senators I'll mention.

One is Tom Har-

Scott: By the way, did you ask, uh, Bill
Frist if he had a PhD in statistics?

Donald Berry: No, I didn't.

That would've been…

Scott: Two,

Donald Berry: I mean, it, this is,

Scott: uh, two

Donald Berry: this is very formal.

I mean, you don't speak
unless you're, you know

Scott: Yeah.

So then Tom Harkin, you,

Donald Berry: Yeah, Tom Harkin,
a very, very nice gentleman.

Um, and he asked a, a question, and
then he gave a response to my answer.

Uh, th-this was a-- the, the meeting
was ostensibly about, uh, a recent,

uh, article that had been published
by the Cochrane Collaborative,

uh, that essentially dissed, uh,
screening for women in their 40s.

Um, and, um, had done, you know, t-
had taken a subset of the, of the,

uh, data that was available and showed
that in that subset it was effective.

Uh, so explaining this to…

That was one of my jobs, was to
explain this to the, to the committee.

Uh, and so Harkin says, in response
to what I said, uh, "Well, Dr.

Berry, uh, I met a woman yesterday
in Iowa, uh, and she told me

that she would rather be a false
positive than a false negative."

Now, my answer was and is

that's fine, but she
doesn't have the choice.

So in the one case, you do have cancer,
the other case you don't have cancer, and

in both cases you get the wrong answer.

Uh, and the other one, so there were like
a dozen senators, uh, asking me questions,

and the last one was Hillary Clinton.

Um

And she was the only one of the
people there that asked reasonable

questions, that listened to the
answer, that responded to the answer.

She gave her, uh…

You know, she, she eventually, in her
discussion, fell into line with, uh,

where they, where they were going.

I mean, how do you, how do you
vote against this when everybody

thinks it's the cat's meow?

Um, but she was clearly understanding
the science and understood what I said.

And so I, I thought she was the
brightest person in the room

Scott: That's fa- that's fascinating.

So I, uh, largely the others
are grandstanding and making

a statement and, and, and she

Donald Berry: Yeah.

Scott: asked

Donald Berry: Uh, this wasn't,

Scott: questions.

Yeah,

Donald Berry: about her.

This was about her, uh, constituents,
but it was about the science

Scott: Okay, so was that the time, by the
way, after this that there was a 98 to

zero vote that, uh, mammograms help women?

They're, they're dictating the
state of nature, or did that

happen after the consensus

Donald Berry: No, that was
a- after the consensus panel

Scott: Okay.

Okay.

Okay.

So following this, by the way, um, uh,
there's something called the Berry effect.

What, what has been
coined the Berry effect

Donald Berry: Um, so

The recommendation that eventually
came out of this for the task force,

uh, was essentially event-- Well,
ev-eventually, it changes over time.

Uh, but, uh, uh, eventually,
uh, was that we shouldn't…

This, this shouldn't be
in all women in their 40s.

Uh, you should rec-- Physicians and
other medical, uh, uh, personnel

should explain, uh, the circumstances
and the pros and cons and let-- and

help the woman, uh, make a decision.

Um, and what they found was that
women in their 40s, uh, when they

collected data on this, uh, fewer
were, um, uh, getting mammograms.

And so I don't know that this was,
uh, uh, uh, renowned a-across the, uh,

government, but somebody at least, uh,
described this as a, as a Berry effect.

Uh, and I, I don't think it was
a positive statement about Berry

Scott: So, uh, my wife Tammy um,
reading a James Patterson novel

Murder Games, uh, a thriller, and
comes across the following passage.

And this is, uh, I'm
quoting from the book.

Um, and they're, they're talking
about the individual here is Dr.

Dylan Reinhart is a Yale undergrad,
PhD in psychology, also from Yale,

three-year research fellow, University
of Cambridge, then another PhD,

this time from MIT in statistics
with a focus on Bayesian statistics.

So, uh, and then the, the, the lawyer here
paused to look up and says, "Am I supposed

to know what that is, Bayesian inference?"

And the response is, "Bayesian inference
is why most women shouldn't have routine

mammograms until they're 50," I said.

So she, she comes across
this rather sporadically.

So, uh, even in a James Patterson
novel, the Bayesian statistics was

attributed to this decision for women
and, and mammograms in their 40s

Donald Berry: Yeah.

So, uh, uh, I, I assume it's the
meta-analysis that I published and

that you did the, uh, calculations
for, uh, in the Journal of the

National Cancer Institute, uh,
where I did a, a meta-analysis.

Can you show that picture?

The one, the one figure that shows the

Scott: I, we, we, I, I, I, that's, I,
I should know how to show pictures.

I don't.

Uh, so I don't have the ability
to show a picture, and most people

are consuming this audio only, so
that's a ch- your challenge here

Donald Berry: Okay.

So if you look at the

There's a meta-analysis that I did
for these, um, women in their 40s.

Uh, the-- there were, uh, uh…

Depends how you count, but there were,
like, five, uh, sites in Sweden, five

different, uh, uh, clinical trials.

Uh, and this is taking the subset
of patients who were in their 40s,

uh, which were reported, uh, for the
conference and for, uh, continued.

It was a subset.

Uh, there's two Canadian trials.

One was for women in their 40s,
the other women in their 50s.

Um, there was one trial
from the United States.

It's a HIP trial, the, uh, Health
Insurance Plan of New York trial.

Um, there was, uh, uh, one trial from UK.

Um, and eventually there was another
trial not, uh, presented at the

conference, uh, at the, uh, development
conference, um, that, uh, uh, was

exclusively women in their 40s.

In fact, you could only be
randomized if you were 40

years old, and then you follow.

It's called the AGE trial.

Um, and what the, the-- what you
would see is, uh, two curves.

One is the mortality for screening,
one is the mortality for the

control, uh, who were assigned to
not get treated, not get screened.

By the way, there were, um,
violations of that, and that's one

of the, you know, statistical issue.

Um, and if you take the area between
those, which I did, uh, you find--

There, there was, uh, the primary
analysis was on the, uh, reduction of

hazard of death, uh, and it was 18%.

So if you take and look at two curves,
one has 18% better than the other,

um- You might say, "Well, what does
this mean to the individual woman?"

Well, uh, so I calculated that by
taking the area between those two

curves and say, on the average, a
woman, uh, who gets screened, if

you believe the data, um, is 1.4

days.

Now, I took a lot of heat from
putting this in there and for

advertising this because people said,
um, that this minimizes the effect.

And I said, "No, this
doesn't minimize the effect.

It is the mean effect.

I'm not minimizing anything."

Uh, so then the, the question
turns to, as Scott indicated,

so what are the negatives?

Um, and, uh, he listed some, uh…

A couple that he didn't insist,
uh, uh, uh, that he didn't, uh,

show was the fact that there are
more people with cancer for longer

Now maybe that's okay.

Pe- women are willing to accept
that if there's a benefit.

But is it, uh, enough in view
of the, the modest benefit?

Is it worth living for
cancer-- with cancer?

Is it worth getting treated for cancer,
um, with questionable, uh, results?

Um, and when I presented, uh, something
I haven't, uh, talked about, uh,

haven't, uh, even mentioned it to Scott.

I, I presented before this meeting at,
um, the, the, at the Senate and back

in, uh, 1997 or 1998, I presented to
the-- to Klausner's advisory committee.

Um, and I presented this issue,
you know, that the-- that you're,

you're making more cancer.

And in-- and by the
way, uh, in terms of…

Well, let me put that on the
back burner for a minute.

Um, I wanna talk about the Obam-
Obama effect and death squads.

Um, the, um…

When I presented it, I had a hard time
explaining why this was a negative

Um, that you, you know, you, you are
now known-- you now are a survivor, and

people are proud of being a survivor.

Um, but, uh, that it, it, it
didn't ring, uh, for them.

Um, but eventually, I said it enough
that it came to be, you know, part of

the rationale for the, uh, US Preventive
Services Task Force and one of the, one of

the harms associated with, uh, screening.

Uh, this thing that I, I pulled
back on, and we probably won't

talk about it except for here.

Um, in 2009, 2010, the US, uh, PC,
uh, the, the, the task force, uh,

had the s- the same recommendation
that the Consensus Development

Conference, um, uh, uh, had recommended.

Namely, uh, we're not gonna recommend…

This is a, a low rating, that it's,
um, not something that you would, um,

uh, you know, recommend that all women
in their 40s get mammograms, uh, but

leave it to the, the woman's decision.

Um, that in 2009, when the task
force announced that, uh, they were

accused of being a, um, they're using
our modeling, uh, which is something

that we ha- we haven't talked about.

Scott: Yeah

Donald Berry: th- that this was an
example of Obama's death squads.

You know, we're sacrificing
people, uh, to save money.

Uh, and it is true that it saves
money, but that's a side effect.

It's never been my, uh, rationale,
and I, I, I can speak with some

authority that it was never the Obama
administration's, uh, uh, authority

Scott: Yeah, so let's, uh, CISNET I
think is a different conversation.

So now looking back years in the
rear view mirror, any regrets?

Any, any thoughts about
the whole activity?

Donald Berry: And I'm
not in Lubbock, Texas.

You know, Lubbock, Texas in
my view, rear view mirror.

Scott: Yeah.

Donald Berry: Um

I-it's-- We mentioned this,
this guy Rick Klausner again.

Uh, he left the NCI,
uh, shortly afterwards.

He was the, uh, director of the
NCI, and he was instrumental

in founding the company GRAIL

Um, and GRAIL is a company that if you
haven't read about, uh, you should.

It's a pan-tumor assay that, um

That detects any kind of cancer.

So it does a test and says, uh, "You have,
or you will shortly have colon cancer

or breast cancer or lung cancer," or,
you know, it would identify the cancer.

Um, and so he, uh, was advisor to
Illumina, this, uh, mega, uh, computer,

um, uh, company, and they had a s-
uh, you know, uh, uh, they, they

built this company, Grail, uh, and
invested, uh, a billion dollars or more.

And other people, the Gates
Foundation, uh, uh, everybody's

getting onto this, um, uh, hayride.

Um, and it raises questions, uh,
all of which we've talked about

are questions, but many more.

I mean, instead of a false
positive, suppose you, um

Suppose you take the test and it
says lung cancer, and you have

cancer, but it's breast cancer.

Is that a positive?

It raises all kinds of
questions, and I was an advisor.

Uh, uh, Klausner, uh, asked me to be
on the scientific advisory committee

for this because he wanted a skeptic.

Well, he got a skeptic.

Uh, and it w- it-- The story is recounted
by Gina Kolata in The New York Times

about how, uh, they finally got rid of me.

I mean, they fired the SAB and
then hired another one without me.

Um, but I told them that they didn't
know what they were doing, that

they were on the road to economic
disaster, if not, uh, medical disaster.

And that, by the way, has, uh, uh, uh…

It, so in answer to your question, Scott,
it's still with us, and people are just…

I mean, Bill Gates are just

was bowled over by this possibility
of early detection, and we should

have learned a long time ago,
long before screening mammography.

Uh, and there's a, I think, a, a, a
well-known and celebrated example from

Japan in neuroblastoma and neonates that
I won't tell you about, but it te-- it,

it, it tells the story, uh, very neatly.

It's-- Steve Goodman calls it the canary
in the coal mi-- in the gold mine.

It, it, it, it told us what we
should have known, and it continues

to tell us what should be known.

Um, but what I see, unfortunately, uh,
with this, uh, the, the poly, uh, gene

assay of GRAIL, uh, with other things,
there is a great deal of ignorance

in the world that is promulgated by
journals, not all journals, but many.

That is promulgated by people who
are conflicted, by entire groups of

medical doctors who are conflicted by
finding cancer and, you know, getting

into the hospital and into my lab.

Um, so there are many lessons, some
of which some of us have learned, but

it should be a course in statistics.

Uh, epidemiologists study this, but
to, to understand the effects of, uh,

what you see may not be what's real

What you see

Scott: Yes.

Donald Berry: is that screening--
that the screening group do well,

but it's not because of screening,
not necessarily because of screening.

And i-i-it, it would be-- th-there's
no dearth of, of, uh, articles that, um

Do I have time for a minute?

Do I, do I gotta tell you?

S-so, so,

Scott: can do a minute

Donald Berry: um

There was a

Um

an article

That, uh, and in the lay press about
centers, cancer centers that were

advertising their cancer outcomes

And mine was one of them

Scott: Your institution

Donald Berry: institution was one of them.

And

Uh, I argued with my institution
and I said, "You know, what you're

doing is you're saying in non-small
cell lung cancer that if you adjust

for stage, um, you are s- y-you are
succumbing to a well-known effect.

It's called the Will Rogers effect.

Um, it is…

You can, you can be better stage
by stage of disease than whatever

control you have, even though in total

It's worse.

So this is a, uh, Simpson's paradox.

Uh

Scott: So the Will Rogers originally
this came from where he said that

when, when there was the large movement
from Oklahoma to California, said

that this made both states smarter.

Donald Berry: Yeah.

In-in-increase-- When the Okies moved
from, uh, Oklahoma to California,

it increased the IQ of both states.

Scott: Yeah.

Donald Berry: and so

Scott: by, by the way, I know with
my Australian friends that they say

this about New Zealand, as well.

So the, this, this has lots of
names, this effect, uh, whi- which

we refer to as the Will Rogers

effect

Donald Berry: So I tried to convince
them to take down their, um, their ad

that they had on their website for,
you know, stage by stage or better.

And I happened to tell that to Gil Welch.

Gil Welch is a person from
Dartmouth, um, uh, who publishes…

A brilliant guy who publishes
on, uh, these kinds of things.

And I-- So I told him that I was arguing
to take down this ad, and he said, "Don't.

It's a wonderful piece of, uh, my, my,
uh, class that I'm giving on this stuff.

And, and I need s- I need junk like
that to, you know, to make a point."

Scott: Yeah

Donald Berry: So maybe the Gil Welches
of the world will, uh, have an effect.

Uh, I didn't get them to take it down
right away, but eventually they did

Scott: Right.

More, more impact.

So, uh, appreciate you going back
to this story, going back in time.

Uh, not Lubbock, but, uh,
uh, back to the story.

And, and as, as your last response is,
this is an issue that continues, uh,

today, if not much more challenging with
the new, uh, diagnostics of all sorts.

Uh, this, these issues, um, uh, uh,
are, are s- even more important today.

So it's a, it's a very cool story of
a huge amount of impact and, and the,

the impact that statisticians can have.

So thank you for going
through that story again

Donald Berry: Thank you.

It's been my pleasure

Scott: And everybody appreciates
you joining us, and until next

time, we'll be here in the interim