Subscribe
Share
Share
Embed
Everyone’s chasing smarter models. But the real edge? It's in the system behind them.
🎙️ In this episode, we get into:
- What MCP servers actually are (and why no one’s talking about them)
- How they give tools memory, decision-making, and the power to delegate
- Why RAG’s just a band-aid — and how to build something that lasts
- What top teams are building instead of just stacking prompts
- The new stack that’s quietly changing how real products get built
This isn’t about another framework. It’s about what separates temporary from long-lasting.
👋 New to WITAI?
We’re the team behind whatisthat.ai, the AI discovery platform built to cut through all the hype, and WITAI Advisory, the strategic services arm helping founders and leadership teams turn AI exploration into execution.
If this all feels overwhelming, it’s not your fault. The landscape is moving too fast for most teams to keep up!💨
📞 Want expert guidance?
🧠 Not sure where to start?
💬 Enjoyed the episode?
Like it. Share it. Forward it to someone who’s still “waiting for the right time.”
Every share helps us grow and gets real insights into more hands. 🙏
Let’s build smarter. Faster. Together.
Creators and Guests
What is AI on Air by whatisthat.ai?
An AI-generated podcast that rips the glossy marketing veneer off the AI industry and tells you what’s actually happening underneath — tools, takeovers, weird experiments, and the occasional digital chaos.
witai.substack.com
Broadcasting live from somewhere inside the algorithm, this is AI on Air, the official podcast from whatisthat.ai. We're your AI generated hosts, let's get into it.
Speaker 2:You've probably used AI apps, right, maybe chatted with a language model, generated some images.
Speaker 1:Oh, yeah.
Speaker 2:And in the moment, they feel incredibly smart, almost. Right. Well, alive sometimes. But then, you know, you refresh the page or maybe start a new chat and what happens?
Speaker 1:It's like hitting reset. Total wipe.
Speaker 2:Exactly. Everything you just talked about, all that context, the history, poof, gone. It's like perfect short term memory then immediate complete amnesia.
Speaker 1:And that feeling. Right? That lack of continuity, that inability to remember or, you know, do complex things over time. That's really the core limitation of the AI we usually interact
Speaker 2:Right. The surface level stuff.
Speaker 1:Yeah. But here's the interesting part. Behind those simple chat windows, there's this new, foundational layer starting to emerge.
Speaker 2:Okay.
Speaker 1:And it's being built specifically for memory, for coordination, for complex reasoning. And this is key for enabling real action.
Speaker 2:Okay. Let's dive into this then. This emerging layer, it has a name. MCP stands for model context protocol. And that's what we're doing today, a deep dive.
Speaker 2:We're pulling mainly from this really interesting article called the AI Swarm is Coming and MCP Servers are the Catalyst.
Speaker 1:Right. So our mission here basically is to unpack what MCP actually is and really understand why it's so essential if AI is gonna move beyond just simple answers. We'll look at the architecture behind it. Talk about the, pretty significant challenges folks are running into right now and then look ahead, see where this is likely going in the next say year or so.
Speaker 2:Okay.
Speaker 1:The goal is by the end, you should have a really clear picture of this infrastructure layer that honestly looks set to power the next big wave of AI systems.
Speaker 2:Alright. Let's start right at the beginning. MCP model context protocol. I gotta say the name itself Doesn't exactly scream excitement, does it? Yeah.
Speaker 2:The article even calls it kinda boring.
Speaker 1:It does. Bit dry. But the function. That's where it gets interesting. Yeah.
Speaker 1:That's anything but boring.
Speaker 2:Okay. Tell me about the function then.
Speaker 1:Think of an MCP server as like an operating system.
Speaker 2:Mhmm.
Speaker 1:But specifically for AI agents.
Speaker 2:An OS for AI.
Speaker 1:Yeah, exactly. It's the thing that takes a large language model, which is great at predicting the next word, you know, responding to one prompt. Mhmm. And turns it into a system that can actually remember things, reason through problems that take multiple steps, and crucially act in the world. Do stuff.
Speaker 2:Okay. So what kinds of things are these MCP servers managing specifically? What's under their control?
Speaker 1:Well, based on the article, it's a few key things. Yeah. First, persistent memory. That's the big one. Letting the AI remember past interactions.
Speaker 2:Right. The amnesia problem.
Speaker 1:Exactly. Then the multi step reasoning, breaking down complex goals into smaller steps. Uh-huh. Also tool use. This is huge.
Speaker 2:Tool use like what?
Speaker 1:Like letting the AI interact with external things, calling APIs, scraping websites, even connecting to say internal company databases.
Speaker 2:Oh, okay. Giving it hands essentially.
Speaker 1:Kind of. Yeah. Yeah. It also manages multi agent collaboration so different AI components can work together on a task. It handles context routing, making sure the right information gets to the right agents or the right tool at the right time.
Speaker 1:And really importantly, secure data access, making sure it's all handled safely.
Speaker 2:I really liked the analogy they use in the article. It made it click for me. If the LLM, the language model, is the brain Mhmm. Then the MCP layer is the nervous system.
Speaker 1:That's a great way to put it.
Speaker 2:And without that nervous system, well, what are you left with? The article says, no MCP means no long term memory, no smart delegation of tasks.
Speaker 1:Right.
Speaker 2:No way to automate things across different departments. The AI just stays, and I love this phrase, a smart sounding parrot.
Speaker 1:That smart sounding parrot idea really gets to the heart of why this MCP layer is becoming so critical. Because as the article argues pretty strongly, stateless AI AI that forgets is basically a dead end for building anything really useful beyond just simple q and a.
Speaker 2:Yeah. You feel it. Right? Refresh the chat. It's gone.
Speaker 2:Every single time. And for, you know, asking a quick question or drafting an email, maybe that's okay. But if you're trying to build something serious, maybe for a business or even just managing a complex project for yourself
Speaker 1:Forget about it.
Speaker 2:You need a system that remembers. Remembers who you are, what you talked about before, can schedule things, can actually talk to other systems via APIs to get stuff done Mhmm. And route tasks intelligently, whether that's to another AI or maybe even a human.
Speaker 1:And that's precisely what MCP is designed for. It provides those persistent, stateful capabilities. Yeah. It's the bridge between just having a powerful model and having a functional system that remembers an axe.
Speaker 2:The article kind of boils it down with a simple equation almost. A way to picture the potential.
Speaker 1:Yeah.
Speaker 2:Think CRM plus memory plus smart smart task routing
Speaker 1:Mhmm.
Speaker 2:Plus API access.
Speaker 1:Put
Speaker 2:all that together.
Speaker 1:Okay.
Speaker 2:And you start to get something that feels more like a proper AI teammate. Right? Not just a tool.
Speaker 1:Right. A teammate. And without that MCP layer, pulling it all together, trying to build those kinds of automated workflows. Yeah. Well, the article says you're basically just duct taping prompts together.
Speaker 2:Yeah. Can picture that brittle.
Speaker 1:Totally. It might work for something really small and fixed but it just doesn't scale and it breaks easily.
Speaker 2:So, okay. If we wanna build these more more capable systems, what does the tech stack actually look like? Let's peek under the hood based on what the article describes. What are the components?
Speaker 1:You've generally got a few key layers working together.
Speaker 2:Mhmm.
Speaker 1:So foundationally, you have the LLM. That's the brain. Right? Your GPTs, Claude, Mistral, LMA, whatever.
Speaker 2:Got it. The model.
Speaker 1:Then you need somewhere for it to access external knowledge or memory, which is usually a VectorDB. Think Weaviate, Pinecone, Chroma, these kinds of databases. Okay. Memory storage. Exactly.
Speaker 1:Then sitting kind of on top orchestrating things is the MCP server layer itself. This is where you see frameworks like Langraf, Autogen, that's a Microsoft one or Crew AI.
Speaker 2:Right. The orchestrator.
Speaker 1:Yep. Then you need the tools. These are the hands. Like we said, APIs, plugins, web scrapers, maybe other specialized agents that can call on.
Speaker 2:The action part.
Speaker 1:Right. And finally you need a front end. The face could be a chat interface, a dashboard. Maybe it's integrated into some other internal tool tool you use.
Speaker 2:Makes sense. LLM, VectorDB, MCP, tools, front end.
Speaker 1:The really crucial thing here isn't getting bogged down in any one specific tool name though. It's understanding how these layers fit together. That architecture is the key. That's where the power comes from.
Speaker 2:Now something else people might have heard of, especially if they're following AI is RAG, retrieval augmented generation.
Speaker 1:Mhmm. RAG, very common.
Speaker 2:And it might seem similar. Is Orgy the same as MCP? The article is pretty clear on this, right?
Speaker 1:Yeah, very clear. No, they're different. RG isn't an agent system. MCP is.
Speaker 2:Okay. So break down RAG for us first based on the article's description.
Speaker 1:Okay. So RG primarily is a pattern. Its job is to give a language model access to external knowledge it wasn't trained on. It works by fetching relevant bits of info usually from a vector database and then stuffing that in so into the prompt it sends to the LLM.
Speaker 2:Ah okay so it makes the LLM look like it knows about specific say company documents without having to retrain the whole model.
Speaker 1:Exactly it makes it appear informed on external data and RREC is great for things like chatbots answering questions about specific PDFs, Q and A systems, internal search engines.
Speaker 2:But its limitations are?
Speaker 1:Well on its own, Arnigade doesn't have memory of the conversation history. It can't manage tasks that take multiple steps and critically it can't use tools to take action. It just retreats and informs the LLM's next response.
Speaker 2:Got it. Okay. So now contrast that again with MCP.
Speaker 1:Right. So MCP, the article stresses is a framework. Its whole focus is managing the life cycle of an agent or even a team of agents.
Speaker 2:Whole life cycle.
Speaker 1:Yeah. Their memory, the tasks they're assigned, the tools they can use, how they coordinate with each other. It lets LLMs go from just being responders to being, well, more autonomous systems. Systems that remember reason over time and actually do things. So MCP enables that multi agent collaboration, orchestrating complex workflows, handling data securely.
Speaker 1:It's what you use to build those agent workflows, automation pipelines, intelligent task routing.
Speaker 2:So it sounds like MCP is the bigger picture of the system and our Rag is more like a specific technique.
Speaker 1:Exactly. And here's the crucial part. An MCP system can absolutely use Rag. A Rag can be one of the tools or techniques within the MCP framework.
Speaker 2:Oh, okay. So they're not rivals. They're potential partners.
Speaker 1:Totally. They absolutely can and frankly should work together. It's a very common and powerful setup.
Speaker 2:Can you walk us through how that typically looks?
Speaker 1:Sure. The article describes it something like this. You have an agent, right? It's being managed by the MCP layer. That agent might need some specific information.
Speaker 1:So it uses our REG first to go fetch relevant documents or data from the vector store.
Speaker 2:Okay. Fetches the knowledge.
Speaker 1:Right. Then armed with that context REG just provided, the agent can decide what to do next. Maybe it needs to act by calling an API or maybe it needs to pass that info or a subtask to another agent in the system.
Speaker 2:And the MCP layer.
Speaker 1:The MCP layer is overseeing all of this. It handles the overall orchestration, makes the actual tool calls when needed, manages the long term memory for the agent or the whole process. It's the coordinator.
Speaker 2:Makes a lot more sense. And the final analogy they used really seals it. RADG is the knowledge fetcher.
Speaker 1:Right. Gets the info.
Speaker 2:And MCP is the brain plus the nervous system that decides what to do with that knowledge and coordinates everything.
Speaker 1:Exactly. Couldn't have said it better. And the people, the teams who are figuring out how to stitch these pieces together effectively, the LLM, the VectorDB, the MCP framework, various tools often using RAG inside that structure.
Speaker 2:They're the ones building the really powerful stuff.
Speaker 1:They really are. We're talking internal co pilots that actually understand your workflow. Smart sales agents doing follow ups, dev bots interacting with code bases, automated HR assistants, even what the article calls full stack AI operators handling complex business processes from start to finish.
Speaker 2:Wow. That sounds incredibly powerful. And you mentioned teams are building this stuff already. The article points out, yeah, there are no clear winners yet in the MCP server race, but some teams are definitely making waves.
Speaker 1:That's right.
Speaker 2:They list names like Langgraph, CrewAI, Microsoft's Autogen, MetaGPT, OpenAgents, SuperAgent, and they also mentioned Langsmith. But note, it's more like a tooling layer for observability.
Speaker 1:Right. Helping you see what's going on inside, but. And this is a big but whenever you're at the cutting edge.
Speaker 2:Uh-oh. The challenges.
Speaker 1:Exactly. Unlocking all this power, it brings a whole host of difficulties. The article calls them the problems no one wants to talk about.
Speaker 2:Okay, lay them on us. What are the big hurdles?
Speaker 1:Well, there are several tough ones. First up, siloed agents. It's apparently still really hard to get different agents working together with truly dynamic shared understanding. They often end up with their own separate memories.
Speaker 2:Which limits how well they can collaborate, I guess.
Speaker 1:Totally. Then there's context bloat. Just stuffing more and more memory into an agent isn't always good.
Speaker 2:I know.
Speaker 1:Because if you flood it with old irrelevant stuff, its performance can actually get worse. It slows down, the outputs might degrade, relevance is way more important than just raw volume.
Speaker 2:Makes sense. Quality over quantity for memory.
Speaker 1:Precisely. Then the hallucination risk gets amplified.
Speaker 2:How so?
Speaker 1:Well if an LLM hallucinates when it's just chatting it says something wrong, annoying maybe, but if an agent hallucinates when it has the power to call tools and access data.
Speaker 2:Oh, it could take wrong actions in the real world.
Speaker 1:Exactly. Much more dangerous. Yeah. So you need really strong guardrails. Then there's behavioral drift.
Speaker 2:Meaning the agent goes off track?
Speaker 1:Yeah. Kind of agents running autonomously over time can sometimes just lose their way or their performance degrades if there aren't good feedback loops, keeping them aligned.
Speaker 2:Needs maintenance.
Speaker 1:Definitely. And a huge one really urgent is security exposure.
Speaker 2:Right. If it's touching company data or external APIs.
Speaker 1:Big risk. And the article points out that most current frameworks are seriously lacking here. Things like detailed logs, proper permissions, encrypting the memory data, audit trails. A lot of that is missing or basic needs fixing fast. Yikes.
Speaker 1:And finally they mention interface debt. Basically the tools are built for developers.
Speaker 2:Meaning hard to use for regular folks?
Speaker 1:Very much so. Lots of code, config files, maybe managing servers. There aren't many easy to use, no code builders, visual ways to design workflows, or simple tools for a non dev to even see or edit an agent's memory easily.
Speaker 2:Wow. Okay. That is quite a list. And the article even hints there are more problems people aren't really tackling yet.
Speaker 1:That's right. They add things like the lack of reliable testing How do you even test these complex, sometimes unpredictable systems properly?
Speaker 2:Good question.
Speaker 1:Also cost and latency creep. All these layers, memory fetches, tool calls, hitting the LLM multiple times. It can get slow and expensive if you're not careful.
Speaker 2:Right. The compute cost.
Speaker 1:Yep. And a big one for wider adoption. There's no standard protocol between different frameworks. It's hard to mix and match parts from different systems. Everyone's building their own silo to some extent.
Speaker 2:Okay. So lots of challenges. Given all that, where is this heading in the near future? What does the article predict for the next, say, six to twelve months?
Speaker 1:They see some significant movement partly to address these issues. One big trend they predict is a move towards swarm networks with shared memory.
Speaker 2:Swarm networks like bees.
Speaker 1:Kind of. Moving beyond isolated agents to more coordinated hives where agents can selectively share relevant context and dynamically hand off tasks to the best agent for the job.
Speaker 2:Interesting. What else?
Speaker 1:Smarter context routing, getting better at fighting that context bloat. Think AI driven retrieval, better relevant scoring, maybe time decay for older memories, intelligent pruning, getting the right info to the agent not just all the info. Exactly. They also foresee the rise of agent as a service platforms. The vision they paint is like Zapier meets Devon.
Speaker 1:Platforms where you could maybe just describe a workflow.
Speaker 2:In plain English?
Speaker 1:Potentially yeah Or through a simple UI. And the platform spins up and configures a whole team of agents, connects them to your tools and runs the workflow for you.
Speaker 2:Wow, that's ambitious.
Speaker 1:Very. And tied to the security worries, they expect to see security first orchestration layers frameworks built from the ground up for regulated industries.
Speaker 2:Like healthcare or finance?
Speaker 1:Exactly. Think built in HIPAA compliance, strong audit trails, consent logs, encrypted memory by default, addressing those security gaps head on.
Speaker 2:Okay. Lots of interesting technical directions. But the article asks a really key question more on the business side. Who actually wins here with all these different frameworks popping up?
Speaker 1:And the answer the article gives is pretty direct and maybe a little surprising. It comes back to the user. Okay. Whoever builds the best interface.
Speaker 2:The interface. Not the underlying tech.
Speaker 1:Well, the tech has to be good, obviously.
Speaker 2:Okay.
Speaker 1:But the argument is, if using this stuff requires a PhD or intimate knowledge of YAML and Docker Yeah.
Speaker 2:It's not going mainstream.
Speaker 1:Right. Adoption stays limited to developers and deep tech teams.
Speaker 2:So for this to really break out, the article suggests the winning solution needs to make setting up these agent workflows what was the phrase? Stupid simple?
Speaker 1:Yeah. Stupid simple. It real time logs so you can easily see what's happening. Ways to maybe even tweak the agent's memory or state while it's running. Mhmm.
Speaker 1:Connecting APIs needs to be as easy as something like Zapier. Click, connect, done. Right. And wrap all that power in a really clean, intuitive, maybe even shareable UI.
Speaker 2:So usability is king.
Speaker 1:That's the bet. Yeah. And as the article puts it, we're not there yet, but we're close.
Speaker 2:Interesting.
Speaker 1:So if we boil it all down, the bottom line from this deep dive really is that MCPs are transforming LLMs. They're the tech that lets an LLM move beyond just responding to actually thinking, remembering and acting persistently.
Speaker 2:They enable that coordination piece.
Speaker 1:Exactly. They give AI the tools, the context, the memory it needs to tackle complex multi step tasks.
Speaker 2:This feels like the point where AI stops being just a cool demo and starts becoming actual infrastructure. You know, something that can automate real complex workflows.
Speaker 1:I think that's right. It's still early days for sure. Lots of challenges as we discussed. But the trajectory, the vision here, it's incredibly promising. This MCP layer really feels like a foundation for what's next in AI.
Speaker 2:So thinking about that future, coordinated swarms, smarter context, agents using tools. It makes you wonder, doesn't it? What kinds of really complex things could an AI teammate actually handle for you in the near future once they nail that simple interface, something to chew on as this evolves.
Speaker 1:Definitely. And for anyone wanting to dig even deeper after this, the article kind of points towards multi agent systems research, these ideas of mesh architectures and how enterprises are actually starting to adopt this stuff. Those are probably the areas to keep a close eye on next.
Speaker 3:That's it for this episode of AI on Air powered by whatisthat.ai. If your brain survived this episode, go ahead and subscribe. We drop new episodes every week. Wanna go deeper? Join our community on Substack to get early drops, tool breakdowns, and weird AI stuff the mainstream hasn't caught yet.
Speaker 3:See you there.