{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"The Experimentation Edge","title":"The metric Stitch Fix says every experimenter should chase","html":"","width":"100%","height":180,"duration":1242,"description":"SummaryIn this episode of The Experimentation Edge, GrowthBook CMO Ashley Stirrup sits down with Nick Beyler, data science manager at Stitch Fix, where he leads the decision and insights team and owns the company's internal experimentation platform. Nick shares why the metric he most wants is the one he can't measure yet, a North Star that predicts a client's long-term value from their earliest behaviors, and why the most impactful experiment learnings tend to come from adoption friction rather than product bugs. He makes the case that if you're only testing winners you're not taking enough risks, explains how guardrails make that risk safe, and looks ahead to a new in-house platform and the promise of agentic AI. It's a practical, statistician's-eye view of experimentation for product managers, data scientists, and engineers building serious testing programs.\n\nChapters00:00 Cold open and welcome to the show01:45 What Stitch Fix actually does04:15 Balancing AI with the human stylist05:15 From public policy to the A/B testing adrenaline rush07:15 Inside the weekly experimentation review group08:45 The AI style assistant and listening to qualitative feedback10:45 Why adoption friction beats product bugs13:45 Testing for losers and building guardrails15:45 Keep rate, successful fixes, and the holy grail metric18:15 The new platform and the promise of agentic AI\n\nTakeawaysThe most impactful experiment learnings usually come from adoption friction, not product bugs. By the time a big feature reaches A/B testing, it's often already a winner, so the open question is how and where to introduce it.A losing test is a finding, not a failure. If every experiment wins, you're not taking enough risk to learn anything new.Guardrails and stopping criteria are what make risk-taking safe, especially when the experience is as personal as shopping.The most valuable North Star metric is the one you can't measure yet, long-term client value, and causal-inference modeling helps predict...","thumbnail_url":"https://img.transistorcdn.com/D9kLs0HSsqR4ttk_5ESEdC1jX-wmD76GK-OHmb3a9B8/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS80YTFk/MGU1MjJlODhlNjJh/MTdlZTZkN2Q1ODY5/OTdjYy5wbmc.webp","thumbnail_width":300,"thumbnail_height":300}