{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"The Experimentation Edge","title":"Why US Bank considers missing even 1% of customers unacceptable","html":"","width":"100%","height":180,"duration":1356,"description":"Summary\nHow does a major bank scale experimentation when even one percent of customers missing an experience is unacceptable? Vijay Lal, Lead Product Manager for Experimentation at US Bank, joins host Ashley Stirrup, CMO at GrowthBook, to share how his team made their experimentation platform self serve for non technical marketers, how a login widget experiment led to a two second fallback that accounted for every customer, and why metrics should be driven by hypotheses instead of handed down by leadership. They also dig into where AI genuinely saves time in experiment analysis, why a human in the loop is non negotiable, and what real time personalization means for the future of testing. This episode is for product managers, data scientists, and experimentation leaders, especially those working in regulated industries.\nChapters00:00 Cold open and welcome00:40 Vijay's path from Comcast to financial services03:16 Making the experimentation platform self serve05:08 The login widget experiment and the two second fallback08:59 Documenting learnings from every experiment10:38 AI in experimentation and the human in the loop12:39 Advice for new product managers15:24 Hypothesis driven metrics18:25 Real time personalization and agentic AI20:11 Democratizing experimentation with responsibility\nTakeaways-Self serve experimentation lets a small central team support a huge testing volume, but it only works with continuous training and guardrail metrics attached.-In a regulated industry, every customer must be accounted for. Even one to two percent of users missing an experience is unacceptable.-A simple fallback, like a two second load rule, can save an ambitious experiment without sacrificing coverage or security.-Metrics should be driven by the experiment's hypothesis, not chosen by leadership in a silo. Pair a primary KPI with secondary KPIs for return behavior.-AI saves real time in experiment analysis, but a human in the loop must validate anything AI produces before it...","thumbnail_url":"https://img.transistorcdn.com/D9kLs0HSsqR4ttk_5ESEdC1jX-wmD76GK-OHmb3a9B8/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS80YTFk/MGU1MjJlODhlNjJh/MTdlZTZkN2Q1ODY5/OTdjYy5wbmc.webp","thumbnail_width":300,"thumbnail_height":300}