{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"The Experimentation Edge","title":"How Supercell A/B tests 300 million players without breaking trust","html":"","width":"100%","height":180,"duration":1427,"description":"Summary\nOn this episode of The Experimentation Edge, Ashley Stirrup talks with Shan Huang, data scientist on the central experimentation team at Supercell, the Helsinki mobile game company behind Clash of Clans, Clash Royale, Brawl Stars, Hay Day, and Boom Beach. Shan explains how a famously decentralized, creative-first company with 300 million monthly active users runs fewer than 100 A/B tests a quarter and why that restraint is deliberate, how Supercell announces experiments to players in advance and promises make-up events to keep testing fair, and how importable AI skills now let anyone at the company analyze their own experiments, making quality consistency the next big challenge. It's for product managers, data scientists, and growth leaders balancing creative conviction with experimental rigor.\nChapters\n00:00 Intro\n01:10 About Supercell and 300 million players\n02:30 The central team and a decentralized culture\n04:25 Fewer than 100 tests a quarter\n05:40 Sharing learnings across independent game teams\n07:35 Retention as the North Star\n08:35 Onboarding experiments with gems and tutorials\n12:45 Telling players about A/B tests\n15:15 Hypotheses and proxy metrics\n20:45 AI and the future of experiment analysis\nTakeaways\n-Supercell runs fewer than 100 A/B tests a quarter for 300 million monthly players, because the goal is to become more hypothesis driven while staying creative, not to maximize volume.\n-In a decentralized company, a central experimentation team earns its impact by providing the platform, partnering on rigor, and making sure learnings travel across independent game teams.\n-Announce experiments to users before they run; Supercell's community managers tell players what is being tested and why, which turns a skeptical community into a research partner.\n-Promise fairness, not just transparency; players who get the worse variant always receive a make-up event later, because game players come to have fun, not to be disadvantaged.\n-AI-powered self-serve...","thumbnail_url":"https://img.transistorcdn.com/D9kLs0HSsqR4ttk_5ESEdC1jX-wmD76GK-OHmb3a9B8/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS80YTFk/MGU1MjJlODhlNjJh/MTdlZTZkN2Q1ODY5/OTdjYy5wbmc.webp","thumbnail_width":300,"thumbnail_height":300}