{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"Old School / New Tech","title":"E09: How to Talk AI Agents to Find Sneaky Production Bugs","html":"<iframe width=\"100%\" height=\"180\" frameborder=\"no\" scrolling=\"no\" seamless src=\"https://share.transistor.fm/e/d09bf77e\"></iframe>","width":"100%","height":180,"duration":1946,"description":"Host Ran Aroussi (Old School / New Tech) shares a recent “war story” from his agency Automaze: a hard-to-reproduce mobile bug where calls were sometimes dropped only on a client’s devices, which the team couldn’t reproduce for 2–3 weeks. After two long sessions using a Droid Factory setup with agents and a “small council” debate between models (e.g., Fable and Sol), he got the team unstuck, found the issue, and then turned the session logs into an internal guide and public article on his methodology. Key practices include distrusting agent conclusions (treat “preexisting/flaky/unrelated” as hypotheses), demanding proof (including his Proof library requiring video evidence), using extensive unit and end-to-end tests, asking for a numeric confidence level before production, controlling environments, handling merge conflicts with full review/testing, switching models for independent review, and re-spec’ing from scratch to detect drift from the implementation plan.\n00:00 Show Begins\n00:58 The Sneaky Mobile Bug\n02:46 Agents Find The Fix\n05:23 Never Trust Preexisting\n08:29 Proof Over Promises\n10:14 Confidence Level Gate\n12:47 Model Debate Reviews\n17:08 Merge Conflicts Standards\n21:23 Spec From Scratch\n25:49 Ten Rules Summary\n31:30 Token Usage And Wrap Up\n-----\nOld School / New Tech is produced by VarOps.\nSubscribe to the newsletter – the written version of this show, weekly: varops.com\nRead the books – Company-Scale Agentic AI (for operators and the C-suite) and Production-Grade Agentic AI (for engineers): varops.com/books\nWork with us – VarOps installs resident AI inside mid-market companies. It runs where your work already happens, sees the whole operation, and what it learns belongs to you: varops.com/transform","thumbnail_url":"https://img.transistorcdn.com/13nrINn33mdG7k8LoP1Ez_oYKpPxbcHmR2r_rrXw8jQ/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS85YTRj/ZGQ0Y2RkZDdmMGY4/NzdjYjgxNjI4OWQ5/YTdkYS5qcGVn.webp","thumbnail_width":300,"thumbnail_height":300}