Conversion rate went up, but did profit? Most testing programs optimize for one metric in isolation, and that's exactly where they go wrong. In this episode, Amanda Siegel (Director of E-commerce at Scale Media) breaks down why she stopped calling what she does "CRO" five years ago and started treating testing as a tool to validate business strategy, not just chase website wins.
In this episode, Alex and Amanda explore what happens when you balance conversion rate, AOV, and subscription adoption in every test instead of picking one, why the old UX tricks (spin-to-wins, slashed prices, CTA color swaps) are losing their edge, and how to get buy-in for short-term metric dips that lead to long-term growth.
Timestamps:
0:00 - Intro and why Amanda calls herself "a little crazy"
1:20 - The limits of traditional CRO and flat testing
3:00 - What's changed in the last 10 years of e-commerce UX
4:17 - Consumers are catching on to optimization tricks
6:10 - Why reducing discounts is actually improving conversion
7:36 - Shifting from product sales to business strategy (bundles, subscriptions)
9:02 - No silver bullets: what works for your business vs. someone else's
10:20 - Conversion rate is one-dimensional and easy to game
11:43 - The growth experiment mindset: testing for business growth
13:58 - Intentional trade-offs and why short-term losses can be worth it
15:10 - The two most important questions in commerce
16:19 - Getting buy-in when a metric temporarily looks bad
19:15 - "Testing is a tool, not a job title"
21:14 - Diminishing returns of small, isolated tests
22:58 - "Don't plan your tests. Test your plan."
24:03 - Amanda's advice: be bold, back-plan from ideal outcomes
25:39 - Why testing has to be cyclical, not one-and-done
27:06 - Where to connect with Amanda
Topics covered:
Why conversion rate optimization (CRO) is too narrow for modern e-commerce
Balancing conversion rate, AOV, and subscription adoption in a single test
How consumer behavior has evolved and why old UX tricks are losing effectiveness
The counterintuitive finding that reducing discounts can improve conversion
Using A/B testing and experimentation to validate business strategy, not just website changes
Getting executive buy-in for short-term metric trade-offs
The growth experiment mindset for Shopify brands
Why testing needs to be cyclical and revisited over time
---
Want to think bigger about your testing program? Join GEM Academy for free courses and a community of e-commerce operators sharing what actually works: https://www.skool.com/intelligems-academy-1535
Connect with Amanda Siegel: https://www.linkedin.com/in/amandadsiegel/
Connect with Intelligems:
Website: https://intelligems.io
Blog: https://intelligems.io/blog
LinkedIn: https://www.linkedin.com/company/intelligems