{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"The Experimentation Edge","title":"Atlassian's Andrew Willingham on the Talent Product That A/B Testing Turned Around","html":"","width":"100%","height":180,"duration":1354,"description":"This episode of The Experimentation Edge explores how A/B testing, feature flags, and user research transformed Atlassian's talent product after it failed with its first users. Andrew Willingham — 11 years at Amazon, now Head of Legal and People Products at Atlassian — shares how product experimentation works when you can't test at scale, why your customer and your user are not the same person, and how the metrics you choose decide which experiments you can even run.SummaryAndrew Willingham, Head of Legal and People Products at Atlassian, spent 11 years at Amazon before joining Atlassian a year ago. His path from running A/B tests on millions of Amazon shoppers to building talent management software for a few hundred thousand employees forced a fundamental shift: when you can't run tests at scale, you have to sit with your actual users and watch them fail. He shares how building a talent review product for Amazon's HR specialists completely flopped when handed to HRBPs — and why that failure taught him more than any winning experiment. Now at Atlassian, he's applying that same rigor to reimagining hiring processes with AI, testing everything from recruiter screens to interview sequences that the industry has run the same way for decades.Timestamps03:09 From marketing Amazon's mobile app to building HR software for 1.5 million associates  08:19 Why a talent review product loved by IO psych experts flopped with actual HRBPs  11:11 How A/B testing helps product managers escape opinion-based politics  15:25 Testing copy that changes behavior: \"We'll generate that status report for you\"  17:20 The two North Star metrics Andrew optimizes: efficiency and quality  19:05 Khan Academy's metric trap: measuring cognitive engagement, not just completion  21:10 Why product managers resist experimentation — and what changes when you admit you don't know  Takeaways- Your customer and your user may not be the same person — building for HR specialists instead of the HRBPs who...","thumbnail_url":"https://img.transistorcdn.com/D9kLs0HSsqR4ttk_5ESEdC1jX-wmD76GK-OHmb3a9B8/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS80YTFk/MGU1MjJlODhlNjJh/MTdlZTZkN2Q1ODY5/OTdjYy5wbmc.webp","thumbnail_width":300,"thumbnail_height":300}