{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"The Experimentation Edge","title":"How Principal A/B tests on customers who don't exist","html":"","width":"100%","height":180,"duration":1894,"description":"SummaryWhat if you could A/B test on customers who don't exist before spending a single live impression? Erika Dunn, assistant director of data science at Principal Financial Group, joins host Ashley Stirrup, CMO at GrowthBook, to share how she built synthetic digital audiences entirely in house: profiles shaped by real data that rank content by likelihood of engagement, whose first live A/B test selection just beat the control. They also dig into the looping metric her team built in SQL to find where customers get stuck without heat-mapping tools, why testing gets watered down into \"let's try something\" at large companies, and how a center of excellence that shares wins and losses defeats the \"we tried that years ago\" reflex. This episode is for product managers, data scientists, marketers, and experimentation leaders, especially those working inside large, risk-averse organizations.\nChapters00:00 Cold open and welcome01:40 Erika's path from quantitative psychology to experimentation03:35 When testing gets watered down06:50 Experimentation in Principal's marketing space09:55 The looping metric that finds stuck users12:45 Building synthetic digital audiences15:15 The first synthetic audience A/B test wins22:30 The PDF lesson and meeting customers on mobile25:15 North star metrics and the right contact cadence28:05 Where experimentation goes next with AI agents\nTakeaways- Synthetic digital audiences let teams rank 20 content options by predicted engagement before spending a single live impression, and Principal's first synthetic selection beat the control in a real A/B test.- A looping metric built from web behavior data can find where customers get stuck without heat-mapping tools: watch how often users cycle back to the same page within tight time windows.- Testing gets watered down when \"let's try something\" replaces a control group; a little pre-planning gets far more out of every experiment.- A center of excellence that shares wins and losses turns tribal...","thumbnail_url":"https://img.transistorcdn.com/D9kLs0HSsqR4ttk_5ESEdC1jX-wmD76GK-OHmb3a9B8/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS80YTFk/MGU1MjJlODhlNjJh/MTdlZTZkN2Q1ODY5/OTdjYy5wbmc.webp","thumbnail_width":300,"thumbnail_height":300}