Subscribe
Share
Share
Embed
The drugs were killing people. In April 1989 the safety board stopped the Cardiac Arrhythmia Suppression Trial. Testing an assumption exposes you to temporary downside to discover what actually works.
Show Notes
The drugs were killing people. In April 1989 the safety board stopped the Cardiac Arrhythmia Suppression Trial. Testing an assumption exposes you to temporary downside to discover what actually works.
Featuring: Annie Duke, Nassim Nicholas Taleb, Stewart Brand, Lewis Carroll, Orville Wright, Otto Lilienthal, Ignaz Semmelweis, Roberto Goizueta
Brought to you by:
Trends.vc: Discover new markets and opportunities.
HeadsUp: Know when your competitors move. Before your customers do.
Multi: Turn any content into a YouTube video.
Chapters:
(00:00) The Cardiac Arrhythmia Suppression Trial gets halted
(00:44) A high-converting email page and the cost of split tests
(01:27) Why leaving a working system alone feels safer
(01:38) Leigh Van Valen and the Red Queen principle
(02:24) Most experiments pay nothing
Where to find Dru:
I write an essay daily. Get them free
Follow me on X
Watch my video essays on YouTube
Follow Dru's Notes on Apple Podcasts or Spotify
Subscribe to Dru's Notes for a new episode every day.
Creators and Guests
NT
Guest
OW
Guest
OL
Guest
RG
Guest
What is Dru's Notes?
One mental model each day in a few minutes. Habits, business, building products and thinking clearly. From Dru Riley, founder of Trends.vc.
"Error is the engine of evolution."
Stewart Brand
The drugs were killing people.
In April 1989 the safety board stopped the Cardiac Arrhythmia Suppression Trial.
For years, cardiologists prescribed encainide and flecainide to suppress abnormal heartbeats after an attack. The logic seemed airtight: if irregular rhythms trigger sudden death, suppressing the beat should save lives.
The study tested that assumption across more than 1,700 patients.
When the safety board compared the double-blind columns, 56 patients on the active drugs died, compared to 22 on the placebo.
Halting the trial stopped a lethal practice that was quietly killing thousands of people each year.
The stakes in medicine are life and death, but the same rule applies to any system. Testing an assumption exposes you to temporary downside to discover what actually works.
I face a low-stakes version of this tradeoff on the Trends.vc email join page.
The page converts above 20%. Pushing for a higher conversion rate means running split tests.
When a baseline is already high, most new variants perform worse.
Each test routes visitors to an unproven design. Some variants shed subscribers before we restore the original baseline.
The short-term numbers look worse.
Short-term downside risk is the price of testing. Running an experiment means accepting temporary dips to discover what actually works.
When an offer or routine is already working well, your natural instinct is to leave it alone. Testing carries immediate downside risk. Why touch something that works?
In 1973 a biologist named Leigh Van Valen looked at extinction rates across the fossil record. Within any group of organisms, the probability of extinction stays roughly constant regardless of how long a species has already survived. Age buys no safety.
His explanation was simple: competing species keep evolving, so improving only keeps you level.
Major journals rejected the paper.
So Van Valen founded a journal called Evolutionary Theory and made his manuscript the first paper it published.
He named the principle after the Red Queen in Lewis Carroll's Through the Looking-Glass.
The Red Queen forces Alice to sprint in place: "Now, here, you see, it takes all the running you can do, to keep in the same place."
Standing still guarantees that entropy and competing operators will overtake you.
Most experiments pay nothing. Most attempts drag your metrics down before you find what works.
Each attempt at better opens a door to worse. You pay the short-term drop to find the rare change that compounds.
What working routine will you test to make it better?
In the autumn of 1901, Wilbur and Orville Wright were ready to abandon flight. They built gliders strictly according to Otto Lilienthal’s widely accepted aerodynamic lift tables, yet each launch at Kitty Hawk nose-dived into the dunes. Desperate for answers, they built a six-foot wind tunnel out of an old starch box and a bicycle hub in their Dayton shop. Over two months, they tested more than 200 miniature wing shapes. Testing proved that the world's standard aviation tables were catastrophically wrong. Breaking the established dogma in their shop gave the Wright brothers the only accurate lift measurements on Earth.
In 1847 Vienna, Dr. Ignaz Semmelweis noticed a terrifying disparity across the general hospital’s two maternity clinics: women admitted to the doctors' ward were dying of childbed fever at nearly five times the rate of the midwives' ward. Suspecting that doctors carried cadaverous particles from the autopsy room to the delivery ward, Semmelweis ordered each doctor to wash their hands in chlorinated lime before examining patients. Maternal mortality plummeted from 18% to under 2%. The test challenged medical hierarchy and outraged senior physicians, but testing the uncomfortable hypothesis uncovered the foundation of modern infection control.
On April 23, 1985, Coca-Cola announced it was replacing its 99-year-old formula with New Coke. The company conducted over 200,000 blind taste tests proving that consumers preferred the sweeter taste of the new recipe over both Pepsi and original Coke. But when the new cans hit store shelves, 400,000 angry calls flooded Coca-Cola headquarters. Sales cratered and protests broke out. Just 79 days later, CEO Roberto Goizueta brought back original Coke as Coca-Cola Classic. The $30 million gamble looked like corporate self-sabotage, but taking the risk revealed an asset no survey could measure: customers were attached to the brand's cultural identity, propelling Coca-Cola Classic to its highest market share in history.