DEV

The "30% faster" AI productivity claim looks compelling on paper — but 2025–2026 research from DORA, METR, McKinsey, and others tells a far more complicated story. This episode breaks down what the numbers actually mean for real software budgets.

Show Notes

The promise of AI-driven development savings is everywhere in vendor pitch decks, but the peer-reviewed data behind those claims paints a far more nuanced picture. This episode of DEV.co digs into what the most rigorous 2025–2026 research — from DORA, METR, Stanford, McKinsey, Veracode, and GitClear — actually says about where AI speeds teams up, where it quietly adds cost, and how to build a realistic ROI model before signing off on a major software investment.

Drawing on the 2026 AI development ROI research and statistics compiled by the DEV.co team, the episode covers:

  • The perception–performance gap: METR's randomized controlled trial found experienced developers took 19% longer with AI enabled — while reporting they felt 20% faster. That 39-point gap is a critical warning for anyone pricing a project on team sentiment alone.
  • Where the headline numbers hold — and where they don't: The widely cited 20–30% productivity lift is real, but only on greenfield, low-complexity, well-typed work in popular languages. On high-complexity or legacy-heavy tasks, McKinsey's task-level data puts gains under 10%, and sometimes negative.
  • Three build scenarios on a $500K project: Headline-only accounting, realistic mixed-work accounting, and brownfield-heavy accounting produce wildly different outcomes — from $150K in apparent savings to near-zero net benefit with rising three-year maintenance costs.
  • The four hidden cost lines: PR review overhead, security remediation (Veracode found 45% of AI-generated code introduces an OWASP vulnerability), technical debt from reduced refactoring (GitClear tracked a doubling of two-week code churn since 2020), and quarterly tooling-migration costs — none of which typically appear in a vendor's statement of work.
  • How to structure AI contracts properly: Price each cost line separately with named owners and measurable targets rather than netting everything into a single optimistic headline figure.

Teams building or modernizing complex systems — whether investing in AI workflow automation or navigating legacy software modernization — will find the framework in this episode directly applicable to scoping and contract negotiations. For more on headless architecture tradeoffs in a related domain, check out the episode Next.js + Shopify Headless eCommerce: Architecture, Tradeoffs, and What Most Teams Get Wrong.

DEV.co

RFP.co

What is DEV?

Software and web development from the side that has to ship it and then live with it. Architecture decisions with a cost attached, scoping, technical debt, hiring and vendor selection, and the AI tooling question every engineering team is now answering whether they planned to or not.

Each episode takes one decision — rewrite or refactor, framework choice, build versus buy, how to scope a fixed-bid project honestly — and works through the tradeoffs, including the ones that only show up in year two. Written for engineering leads, technical founders and the people who fund them. Five or six minutes, no hand-waving.

Topics include rewrite versus refactor, build versus buy, scoping fixed-bid work honestly, technical debt you should keep, framework and platform choices, hiring and vendor selection, code review culture, and where AI tooling actually helps.

Produced by DEV.co, web and software development. Full details, services and further reading at https://dev.co