{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"The Experimentation Edge","title":"What Grubhub tests for big product bets before they ship","html":"","width":"100%","height":180,"duration":1303,"description":"Summary\nGrubhub does not run much classic A/B testing. Michal Lenik, Associate Director of Product Design at Grubhub, explains what the company does instead to make sure its big product bets win before they reach millions of users.\nShe joins Ashley Stirrup to walk through Grubhub's playbook: small cohort rollouts that protect core order metrics, merchant round tables, AI built clickable prototypes, and a concept test in the merchant promotions space that sent the team toward a much simpler dashboard than their research predicted. Michal also explains why she follows losing experiments with user interviews, why \"I didn't even notice it\" is valuable feedback, and why every company should decide deliberately whether it de-risks upstream or in a fast follow after launch.\n\nChapters\n00:00 Where design is heading in the AI era\n00:53 Meet Michal Lenik of Grubhub\n02:21 Design as an upstream business lever\n03:48 Small cohorts instead of classic A/B tests\n05:27 The merchant concept test that changed direction\n08:18 Prototyping with AI and merchant round tables\n10:16 Success metrics for product, design and engineering\n11:56 Pairing experiment data with user interviews\n14:26 AI, personalization and the designer's new role\n17:45 De-risking upstream versus shipping fast\n\nTakeaways\n- Grubhub tests big bets by releasing to internal users or small cohorts first, so it can read business impact without disrupting core revenue and order metrics.\n- A concept test with merchants overturned what fifteen research interviews suggested, steering Grubhub to a simpler dashboard and away from a launch that likely would not have won.\n- AI lets Grubhub's design teams put clickable prototypes in front of users early, turning each iteration into a micro experiment.\n- Pairing quantitative results with five to ten user interviews after a test explains why it underperformed, including whether users even saw the change.\n- Both upstream de-risking and fast iteration work. The right choice depends on...","thumbnail_url":"https://img.transistorcdn.com/D9kLs0HSsqR4ttk_5ESEdC1jX-wmD76GK-OHmb3a9B8/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS80YTFk/MGU1MjJlODhlNjJh/MTdlZTZkN2Q1ODY5/OTdjYy5wbmc.webp","thumbnail_width":300,"thumbnail_height":300}