{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"The Experimentation Edge","title":"False negatives are killing your best product ideas","html":"","width":"100%","height":180,"duration":1694,"description":"Summary How do you make a high-stakes product decision when the safe choice is to never test it at all? In this episode of The Experimentation Edge, host Ashley Stirrup talks with Arun Bodapati, director of data science at Twitch, about the discipline behind trustworthy experimentation. Drawing on his experience at Schwab, Uber, and Twitch, Arun explains why false negatives are the most dangerous result a team can produce, what hygiene to nail before you push play, and how Twitch used geo-fenced experiments and causal inference to finally settle a pricing question it had avoided for years. It's a practical conversation for product managers, engineers, data scientists, and growth leaders who want experiments that hold up  and earn executive trust. Chapters00:00 Welcome and introduction01:15 Arun's background and marketing experimentation at Schwab04:15 Uber's mature, experiment-driven culture06:30 Coming to Twitch: from Python notebooks to a shared standard08:30 The pricing problem Twitch had long avoided10:30 Geo-fenced experiments, matched markets, and elasticity13:15 The gifted-subs surprise and testing promotions16:15 The discipline that matters before you push play18:15 Why false negatives are worse than false positives20:05 Enrollment triggers and broad explore experiments22:45 AI, the Kiro tool, and what's next for experimentationTakeaways False negatives are more dangerous than false positives — they get institutionalized as \"we tried that, it didn't work\" and quietly kill good ideas for years.The most valuable experiment work happens before you push play: clear enrollment logic, a plain-English hypothesis, and no optimizing ahead of the test.If an intervention sounds weak when you write it out in plain English, don't run the experiment — you're just wasting time.Run a broad explore experiment first; small, over-narrowed populations lack power and raise the odds of a false negative. Find the responsive segment with heterogeneous treatment effects...","thumbnail_url":"https://img.transistorcdn.com/D9kLs0HSsqR4ttk_5ESEdC1jX-wmD76GK-OHmb3a9B8/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS80YTFk/MGU1MjJlODhlNjJh/MTdlZTZkN2Q1ODY5/OTdjYy5wbmc.webp","thumbnail_width":300,"thumbnail_height":300}