Code and Cognition

Show NotesIn this episode, we cover:00:00 – Agents vs. WorkflowsWhat’s new in LLM-powered agents, and how they differ from traditional, on-rails workflows.01:30 – Pattern Libraries and Agent FrameworksExploring tools like Amazon Bedrock and visualizing agentic workflows.03:00 – LLM as JudgeUsing higher-cost models to validate output from cheaper models. A clever way to balance cost and quality.04:30 – Real-Time Evaluation and Prompt EngineeringHow teams can move from static success criteria to live evaluation systems.05:45 – System Prompt DesignLessons learned from designing effective system prompts and using them the right way in production.08:00 – Tool Use and LLM Decision MakingTeaching LLMs to use internal tools to answer queries—think calendar lookups, database queries, and more.11:30 – End-to-End Testing with Natural LanguageA new generation of testing libraries using LLMs and Playwright to turn plain English into functional tests.12:45 – Linear vs. Branching Workflows in Data PilotHow AI can define and adapt its own steps in data analysis and synthesis.15:00 – Multi-Source LLM QueryingMerging inputs from time tracking, Slack, GitHub, Jira, and more to create richer outputs.18:30 – Human-in-the-Loop PatternsDesigning workflows where humans review and approve AI-generated outputs before final delivery.20:30 – Cursor and AI-Accelerated DevelopmentHow Olio engineers use Cursor for faster iteration while managing tech debt and consistency.23:00 – Heuristics for Managing AI-Created CodeTips for writing better system prompts, rules files, and maintaining consistent output across codebases.27:00 – Infrastructure Throttling and LLM Scaling ChallengesBehind-the-scenes look at orchestrating LLM queries at scale, managing Lambda concurrency, and avoiding AWS throttles.30:00 – LLM Identity & Masquerading ChallengesWhat it takes to make your AI chatbot actually believe it’s your brand—and how Claude and GPT-4 stack up.33:00 – Looking Ahead to 2025The team shares what they’re most excited to build next and how LLMs are becoming critical in everyday workflows.Stay ConnectedSubscribe to Code & Cognition for deep dives into AI, cloud-native development, and the tools shaping modern software teams.Questions or feedback? Contact us at info@olioapps.comOlio Apps can help you build your app: ⁠https://www.olioapps.com/contact-us⁠Hey! We're Scott and Aron. We are software engineers, and business operators running Olio Apps. We help our clients build the best software possible. We're always learning and improving our methods and practices, and we want to share that with you.

Show Notes

Show Notes

In this episode, we cover:

00:00 – Agents vs. Workflows
What’s new in LLM-powered agents, and how they differ from traditional, on-rails workflows.

01:30 – Pattern Libraries and Agent Frameworks
Exploring tools like Amazon Bedrock and visualizing agentic workflows.

03:00 – LLM as Judge
Using higher-cost models to validate output from cheaper models. A clever way to balance cost and quality.

04:30 – Real-Time Evaluation and Prompt Engineering
How teams can move from static success criteria to live evaluation systems.

05:45 – System Prompt Design
Lessons learned from designing effective system prompts and using them the right way in production.

08:00 – Tool Use and LLM Decision Making
Teaching LLMs to use internal tools to answer queries—think calendar lookups, database queries, and more.

11:30 – End-to-End Testing with Natural Language
A new generation of testing libraries using LLMs and Playwright to turn plain English into functional tests.

12:45 – Linear vs. Branching Workflows in Data Pilot
How AI can define and adapt its own steps in data analysis and synthesis.

15:00 – Multi-Source LLM Querying
Merging inputs from time tracking, Slack, GitHub, Jira, and more to create richer outputs.

18:30 – Human-in-the-Loop Patterns
Designing workflows where humans review and approve AI-generated outputs before final delivery.

20:30 – Cursor and AI-Accelerated Development
How Olio engineers use Cursor for faster iteration while managing tech debt and consistency.

23:00 – Heuristics for Managing AI-Created Code
Tips for writing better system prompts, rules files, and maintaining consistent output across codebases.

27:00 – Infrastructure Throttling and LLM Scaling Challenges
Behind-the-scenes look at orchestrating LLM queries at scale, managing Lambda concurrency, and avoiding AWS throttles.

30:00 – LLM Identity & Masquerading Challenges
What it takes to make your AI chatbot actually believe it’s your brand—and how Claude and GPT-4 stack up.

33:00 – Looking Ahead to 2025
The team shares what they’re most excited to build next and how LLMs are becoming critical in everyday workflows.


Stay Connected
Subscribe to Code & Cognition for deep dives into AI, cloud-native development, and the tools shaping modern software teams.

Questions or feedback? Contact us at info@olioapps.com


Olio Apps can help you build your app: ⁠https://www.olioapps.com/contact-us⁠


Hey! We're Scott and Aron. We are software engineers, and business operators running Olio Apps. We help our clients build the best software possible. We're always learning and improving our methods and practices, and we want to share that with you.

What is Code and Cognition?

Discover the Cutting Edge of Gen AI Integration

Step into the world of Gen AI and cloud computing with Code & Cognition — the podcast where industry leaders, developers, and innovators reveal the latest breakthroughs and strategies in AI.