{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"The Experimentation Edge","title":"Why US Bank considers missing even 1% of customers unacceptable","html":"<iframe width=\"100%\" height=\"180\" frameborder=\"no\" scrolling=\"no\" seamless src=\"https://share.transistor.fm/e/1e47f766\"></iframe>","width":"100%","height":180,"duration":1356,"description":"Summary\nHow does a major bank scale experimentation when even one percent of customers missing an experience is unacceptable? Vijay Lal, Lead Product Manager for Experimentation at US Bank, joins host Ashley Stirrup, CMO at GrowthBook, to share how his team made their experimentation platform self serve for non technical marketers, how a login widget experiment led to a two second fallback that accounted for every customer, and why metrics should be driven by hypotheses instead of handed down by leadership. They also dig into where AI genuinely saves time in experiment analysis, why a human in the loop is non negotiable, and what real time personalization means for the future of testing. This episode is for product managers, data scientists, and experimentation leaders, especially those working in regulated industries.\n\nChapters\n00:00 Cold open and welcome\n00:40 Vijay's path from Comcast to financial services\n03:16 Making the experimentation platform self serve\n05:08 The login widget experiment and the two second fallback\n08:59 Documenting learnings from every experiment\n10:38 AI in experimentation and the human in the loop\n12:39 Advice for new product managers\n15:24 Hypothesis driven metrics\n18:25 Real time personalization and agentic AI\n20:11 Democratizing experimentation with responsibility\n\nTakeaways\n- Self serve experimentation lets a small central team support a huge testing volume, but it only works with continuous training and guardrail metrics attached.\n- In a regulated industry, every customer must be accounted for. Even one to two percent of users missing an experience is unacceptable.\n- A simple fallback, like a two second load rule, can save an ambitious experiment without sacrificing coverage or security.\n- Metrics should be driven by the experiment's hypothesis, not chosen by leadership in a silo. Pair a primary KPI with secondary KPIs for return behavior.\n- AI saves real time in experiment analysis, but a human in the loop must validate anything...","thumbnail_url":"https://img.transistorcdn.com/D9kLs0HSsqR4ttk_5ESEdC1jX-wmD76GK-OHmb3a9B8/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS80YTFk/MGU1MjJlODhlNjJh/MTdlZTZkN2Q1ODY5/OTdjYy5wbmc.webp","thumbnail_width":300,"thumbnail_height":300}