{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"The Experimentation Edge","title":"Learneo on testing the opposite of every hypothesis","html":"","width":"100%","height":180,"duration":1281,"description":"Summary\nRich Liebling, senior director of engineering at Learneo, joins host Ashley Stirrup to explain the practice that came out of growing Shop It To Me from 50,000 subscribers to one million in nine months: test your hypothesis, and test its opposite. Rich covers the page where cutting text lost and adding text won, why the inverse wins more often than teams expect, and why small focused tests are the only ones where \"the opposite\" means anything. He also walks through translating a million subscriber goal into a target of 300 A/B tests, making a new engineer's second merge request their own experiment, and the different constraints at Course Hero, where competing team metrics were resolved with an early lifetime value model and three-day SQL analyses quietly capped testing velocity. This episode is for product managers, engineers, data scientists, and growth leaders building or scaling an experimentation program.\n\nChapters\n00:00 Cold open and introduction\n01:50 Shop It To Me and the first engineering hire\n02:55 300 A/B tests as the path to one million subscribers\n04:00 Testing as a core value and the second merge request\n07:20 Learneo, Course Hero, and two ways to get access\n08:50 Modeling lifetime value to stop teams competing\n11:10 Testing the opposite of the hypothesis\n14:35 A portfolio of small, medium, and large tests\n17:15 Multi armed bandits and seasonal traffic\n20:35 The flywheel that keeps copycats behind\n\nTakeaways\n- Test the opposite of every hypothesis. At Shop It To Me the inverse won surprisingly often, and even when it lost it proved the variable mattered.\n- Keep tests small and focused. Redesign a whole page and lose, and you learn that version failed but not what to change next.\n- Translate a growth goal into an execution count. One million subscribers is not actionable. 300 A/B tests by year end is, and everyone can influence it.\n- Make experimentation part of hiring and onboarding. Every new engineer's second merge request was their own...","thumbnail_url":"https://img.transistorcdn.com/D9kLs0HSsqR4ttk_5ESEdC1jX-wmD76GK-OHmb3a9B8/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS80YTFk/MGU1MjJlODhlNjJh/MTdlZTZkN2Q1ODY5/OTdjYy5wbmc.webp","thumbnail_width":300,"thumbnail_height":300}