Subscribe
Share
Share
Embed
Future-proofing the humans behind the tech. Follow Phil Gamache and Darrell Alfonso on their mission to help future-proof the humans behind the tech and have successful careers in the constantly expanding universe of martech.
[00:00:00] Liz: I actually do think a lot of agents are not yet ready to be fully human outta the loop, but in 12, 18 months, like they will absolutely be there.
[00:00:08] Rich: 30 vendors are telling you to go turn on the AI for their application, you sort of become like a AI referee.
[00:00:14] Keith: you are playing translator between
[00:00:16] vendor A and vendor B.
[00:00:17] And the agents or orchestration layers within them.
[00:00:20] Aboli: But then you need an orchestration layer eventually for these agents.
[00:00:23] Phil: Whoever controls that door controls what instructions get through, and which agent gets authority over the customer
[00:00:29] Rebecca: and the reason why we call it dispatch affectionately is because this group is going to be accountable for all marketing communications
[00:00:36] Michele Nieberding: but if you're not enforcing that consent as soon as the data is collected, that's a problem
[00:00:42] Anna: We're shifting from being execution focused to now being very design focused
[00:00:46] Tiankai: can shift more towards end-to-end thinking, value thinking, design thinking, all the things to actually make it better.
[00:00:59] In This Episode
---
[00:00:59] Phil: [00:01:00] What's up, folks? Welcome to our four-part series of crawling through the dungeon of MarTech architecture. You've arrived at our final episode, part four, the dispatch tower. We'll tackle the governance chaos that ensues when dozens of vendors simultaneously activate their own autonomous AI agents, why you need a central referee with agent guardrails, how to build a dispatch layer and why you should focus on system stewardship,
[00:01:26] All that and a bunch more stuff after a quick word from two of our awesome partners
[00:03:36] Welcome Back
---
[00:03:36] Phil: Welcome back to the dungeon of MarTech architecture, folks. You've arrived at the final floor. Parts one through three built the foundation of the meaning layer and the data warehouse, the causal memory. Part four is the floor most organizations have been living on the whole time, often without even knowing it Let's finish our descent.
[00:03:58] FLOOR 4: THE DISPATCH TOWER
---
[00:04:05] Phil: Congratulations, MarTech crawler. You've arrived at the final floor of the dungeon. Let's face it, most of your fellow crawlers will never make it this far. They're still lost in the political battles debating why the CRM shouldn't hold product activation data, or they ended up falling prey to the correlation trap and are forever stuck in the boomerang room.
[00:04:25] But you've made it. This floor is the hardest because it combines collaborative political battles, but some serious technical obstacles as well. The layout of this floor is best described as a hot mess of agent spaghetti.
[00:04:39] What Happens When 30 Vendors Turn On AI at the Same Time
---
[00:04:39] Phil:
[00:04:45] Your inbox is probably already full of emails from every platform in your stack, all saying some version of, "Turn our AI agent on today. Activate our MCP today."
[00:04:56] Each one promises autonomous intelligence, and [00:05:00] each one wants permission to write, recommend, to segment, suppress, blah, blah, blah. Each one is operating from their own model of the customer, and none of them can really explain what happens when two different agents from different vendors try to update the same record or message the same person or optimize against conflicting goals.
[00:05:19] Man, like people working in-house right now are dealing with a mess of agents, and congratulations, you are now a referee, whether you signed up for it or not
[00:05:29] Rich: If you're the, the IT leader of a large organization or the marketing leader. And your 30 vendors are telling you to go turn on the AI for their application, you, you sort of become like a AI referee.
[00:05:41] Phil: That's Rich Waldron, CEO of Tray.ai, and he describes how this feels on the ground
[00:05:47] Rich: You have to figure out like, well, do I turn it on for this one and not for this one? And is this one gonna overwrite what occurs here? And how do I like, manage or govern? You know, what is going like is, is all this going into the same [00:06:00] LLM? Do I have to go and maintain that? How do I make sure that somebody doesn't post like a social security number into the wrong place that goes?
[00:06:07] AndSo if I'm sat in it and I say, Hey, I want the marketing ops person to be able to have. Autonomy or control over building agents.
[00:06:15] Do I want 'em to go and do it in a third party application that requires additional governance, or would I rather have a consistent way that all these executions occur in one place? And I think one of the key components or challenges about a lot of these agents solutions is they rely on well built out integrations and they rely on well-built out.
[00:06:34] Uh, integrations that can carry out action. And I think over time we'll see the AI get better at carrying some of that out on its behalf, you know, building these, uh, uh, building connectivity itself. But we're a ways away from that right now. So I feel like there's gonna be a lot of kind of turnover in a lot of software stacks, experimentation with new applications, trying to find quite what fits.
[00:06:55] Phil: Richard's describing the nightmare version of our final floor here. Like [00:07:00] every vendor, shipping agents, and marketing ops, and rev ops, we're all getting stuck deciding which ones are safe, which ones are redundant, which ones do we pick, which ones are risky, which ones qua- conflict with each other. The answer is not let every team turn on whatever they want, obviously.
[00:07:17] It's also not, let's create this like central AI police task force that blocks everything and innovation dies in this committee. The middle path is an operating model, a small central group of central excellence group, if you want, that sets the standards, reviews the risks, creates reusable patterns, um, trains people internally, and lets the business move quickly without turning the stack into this like haunted appliance store.
[00:07:45] Funny enough, I spoke with someone who actually did volunteer for that AI referee job.
[00:07:50] Lindsay: we, uh, formed an AI center of excellence. So that AI center of excellence is led by a chief AI officer, um, within our company.
[00:07:58] Phil: That's Lindsey [00:08:00] Roethlisberger, director of GTM Innovation at Zapier, and also a member of their AI Center of Excellence
[00:08:06] Lindsay: So he, um, sort of heads up that, that central unit, and it's actually a small team. Um, I think it's actually just a f- a few people that are, like, dedicated to this.
[00:08:16] And really what they do is they set up, um, the resources and the frameworks and the guidelines and the enablement. And I am a part of that AI center of excellence as, like, a guild member dedicated to go-to-market. So I'm sort-- It's like a hub-and-spoke model. Um, there's the central team, and then I'm sort of the spoke out to go-to-market, and I have another person who works with me, um- on that within go-to-market as well.
[00:08:46] So the two of us work on, like, making sure that those things translate into go-to-market, making sure that we're sort of boots on the ground about how people are building and what they need in go-to-market, and, like, being that feedback loop back.
[00:08:57] Phil: So it's super cool. At Zapier, her work covers deciding [00:09:00] which AI tools gets activated.
[00:09:02] Um, she helps set, set the standards for what a shared skill should be in Claude Code and keeping track of like what people are building before it fragments into like 100 different unsanctioned experiments.
[00:09:14] Lindsay: Yeah. Yeah. Yeah, the security team is very deeply embedded with the AI Center of Excellence, so they work really hand in hand in developing the policies.
[00:09:23] We have data partners that are embedded in go-to-market and really understand the in and out-- ins and outs of, uh, our go-to-market, like, data architecture, what we need. So they were a really nice conduit in working with the data team, and they actually in part... Or sorry, with the security team. They also, in partnership with the security team, developed our, uh, skill that reviews our skills for data and security compliance.
[00:09:48] So they built a skill that, um, folks reference when they want to push a skill, you know, to a shared state, um, that they worked in partnership [00:10:00] with us to un-- really understand our use cases and what we're trying to do. Um, so yeah, I totally agree. It's really important, I think now to, for, uh, security and data to sort of change the way that they work with teams.
[00:10:13] Phil: So every shared skill at Zapier has to have a named owner. If it's used across a team, someone is accountable for maintaining it, and skills covering go-to-market operations are owned by RevOps.
[00:10:24] New builders who want to ship a skill that runs through a cross-functional workflow, um, need to pair with a RevOps partner to build it. The governance operates as a co-authorship model, and is designed to make sure that what gets built can also be maintained and trusted by other people.
[00:10:41] Lindsay: so it's sort of changing the dynamic where, like, we're becoming more of, like, enablers and, like, guardrails and, like, setting the systems and the infrastructure, but letting people who are really close to the problems figure out how to solve them.
[00:10:55] Phil: I'm betting that we'll see a lot more roles like Lindsay's pop up over the next few years. Probably [00:11:00] still not at the pace of new AI agents, but hey, we can dream in RevOps. It's, it's almost like every vendor wants to be the AI brain, but, um, they're all competing for it and orchestration, but nobody wants to be the adult in the room and figure out this massive problem
[00:11:16]
[00:11:22] Phil: Sometimes the structural version of this problem is worse than the org chart makes it look. Like enterprise AI deployments regularly expose 20 to 30 tools to a single agent simultaneously. Multiple sources say some version of that agent performance degrades sharply, like beyond five to 10 tools per agent.
[00:11:43] And the failure is, in a sense, architecture, right? Like if a human can't say definitively which tool handles a task, the agent can't either. 30 vendors each waiting for you to turn on their AI creates this organizational avalanche of [00:12:00] agents and a configuration that breaks the reasoning layer before a single campaign can even run.
[00:12:06] Keith: I would say that the, this game isn't all that different than what we've been doing for the last 10 years on trying to get these technologies to talk to each other.
[00:12:14] Phil: That's Keith Jones, who leads one of the GTM systems teams at OpenAI, and he's lived this from the operator's chair
[00:12:22] Keith: And it's about coordination, right? If I've got. And let's just use the, the timeless example of marketing automation and CRM, right? Uh, anyone who's listening to this podcast has probably done any number of integrations with any number of maps and any number of CRMs, and we can probably all have a horror story or several of trying to get those two to talk to each other, right?
[00:12:44] But if you do take the right amount of time and understand the, the timing and order of operations. Between the two, right? Like, uh, you know, you know, obviously my, my career has been pretty heavily with Salesforce, right? That's the CRM I've used most [00:13:00] often. Um, Salesforce has a lot of things about it that are pretty, um, routine, if you will, right between the way that it runs its processing queue to the amount of volume it'll handle at any given time, right?
[00:13:13] Um, and in order to deploy Salesforce at scale. You need to know those things like the back of your hand, right? Because otherwise you might start building a solution, whether it's declarative or custom developed that isn't really compatible with the way the the core system runs, right? So I would say that there's really not a lot of apprehension that's warranted to say, like, don't turn on the AI or don't.
[00:13:39] Use that vendor's orchestration layer, but you need to understand it intimately because at the end of the day, what you're doing as a technologist is you are playing translator between vendor A and vendor B and the agents or orchestration layers within them. Now there's always gonna be space for maybe some sort of master orchestration layer.
[00:13:59] That's [00:14:00] then maybe essentially doing that job for you eventually, right. You can't teach something to do something if you don't know how to do it yourself. So I would just say that you've got to know how these layers are, and Ag agentic workflows are operating at a very intimate level.
[00:14:17] Olga: It depends on also the size of the company and what tech stack you're using
[00:14:21] Phil: That's Olga Andrienko, who left a VP role at Semrush to build AI for marketing ops and adds the procurement reality that most architecture discussions politely cites that
[00:14:31] Olga: if you're already like using HubSpot then, and then they have the agenda capabilities, then I would not think of like going and testing out like 10 different smaller companies. Uh, just like, again, because of procurement, because of all the hoops that you need to, uh, jump through. And, uh, and then ultimately this is about, um, even tech technological abilities of the companies to [00:15:00] store data and then just to process data and then secure your data. And, uh, and if, if you're a larger company, then you tend to stay stick to like more secure, larger vendors. And then they will still, like eventually just well catch on and then see what's like working for smaller companies and incorporate that. So I would not go and analyze like 10 different smaller companies.
[00:15:29] I would just wait, well, or I just built something custom. Or I would look at the existing bigger vendors and then just ask them like, do you have this in the backlog when it's coming? And, uh, but if you're unhappy with the current vendor,
[00:15:46] then it's like, it's good time to evaluate and then take this into consideration like what they have as the upcoming releases of the AI agents.
[00:15:56] Phil: So neither of these paths are wrong. They both [00:16:00] ask you to have explicit decisions. Um, but the teams that drift into vendor-led orchestration without thinking about it are the ones who end up most surprised when something breaks.
[00:16:11] Keith: so, you know, there's a precursor to this idea of Symphony that we've, we've open sourced for orchestrating, AI generative code deployment, um, which is harness engineering.
[00:16:20] Phil: here's Keith Jones again at OpenAI and how his team built and open sourced their answer. The foundation for them was called harness engineering. Building the environment a coding agent needs to operate within, with the patterns in place, the standards it must follow, and what it's allowed to touch.
[00:16:37] Keith: . Um, I'm not going to be able to speak to it to the nth degree to the technical level, as I'm sure as your audience knows, I'm not the developer type, right?
[00:16:43] I fit more into that technical product manager profile. That's who I am at my core. Um, but, um, to that point, like, you know, harness engineering at its simplest, right, is building the infra that a coding agent needs in [00:17:00] order to know where to slot things in and how to do certain things, right? And so we have a harness specific to Salesforce.
[00:17:07] But now with Symphony, we're able to set it up so that if we have something on a ticket that is scoped well enough and has enough specificity and enough very tactical requirements, and, you know, we need to do this here and that there, right? Just like you would deliver something to a junior developer where you know they can do the legwork if they have enough guidance and enough prescription.
[00:17:32] And we're not needing to go to the nth degree of like, write the code like this, although there are some elements to it that I'll talk about in a moment. Um, but we've got the ticket, we've got the clarity. Awesome. We put a label on it, which sort of kicks off the process so that it can use the harness and then orchestrate the code and, and start writing it and start drafting a PR that then an engineer reviews and looks at and sees, "Okay, great.[00:18:00]
[00:18:00] I see what it did here. I see what it did there. Oh, it made a mistake." Common mistake. A junior developer would make that mistake. Again, that's what the agent is, right? And so how do we stop that from happening in the future? The engineer goes back in, writes a new skill, says, "Hey, when you're seeing something like this, don't do this.
[00:18:18] Do this instead," right? Roll back the change, run it again. Guess what? It doesn't make that mistake again.
[00:18:25] And that means the next time we have a requirement like that that requires the same specificity of skill in order to maintain our standards- We don't have to go back and write the skill again because it's already there, right?
[00:18:38] And so the model that we essentially have running at that point is essentially we have our two profiles, right? We have our TPM and we have our engineer, and in between both of them, we have essentially an army of junior developers that are those agents writing that code that's being informed by the crisp requirements that the TPM has provided, and then constantly [00:19:00] being reinforced and upgraded by the engineering staff who understands how to make these agents better over time by continuing to reinforce their practices with those different skills, as well as continue to make, you know, changes to the actual harness itself, as well as Symphony to help just inform that infrastructure, if you will.
[00:19:20] Um, and that's, that's the model. Like, that, that's how it works. And it's, um, it's going in the right direction.
[00:19:26] Phil: At OpenAI, that harness is Salesforce specific in their GTM systems world, and layered on top is Symphony, a generative code orchestration framework that connects a well-scoped ticket to agentic code deployment without requiring a developer to write every line of code.
[00:19:44] Here's Keith on the productivity impact and how he's able to put a number on it
[00:19:48] Keith: I've got contractors who within less than 30 days of being with the business are shipping cha-- the same changes as someone who's been here for two years
[00:19:59] Phil: [00:20:00] Hmm.
[00:20:00] Keith: Think about that.
[00:20:02] That That's an acceleration in productivity that has not been heard of in a technical landscape before.
[00:20:09] Phil: So in this case, the harness makes agent authority specific instead of general. The orchestration layer converts a well-scoped ticket into deployed code. The engineer is basically the senior developer who makes the junior agent better over time. And the model only works though if DevOps and, you know, um, some of the more GTM systems ops discipline existed before anyone added the agent.
[00:20:34] there's three factors that should drive that decision. One is technical capability. so like does your team have engineers who can build and maintain custom integrations? If not, vendor-led is maybe your more realistic path. The second one is data sensitivity. How much of your customer data are you comfortable routing through direct third-party model?
[00:20:54] Um, the more sensitive the data, the stronger the case for API-based control. And the third one is [00:21:00] stack stability. If you're adding or replacing tools every six months, vendor native integrations break constantly. An IPaaS or custom orchestration layer insulates you from that churn. Most teams will land somewhere in the middle, maybe.
[00:21:13] Like vendor-led for low sensitive, data and like high change workflows, and then API-based for anything touches sensitive data or requiring a ton of stable governance
[00:21:24] Boss Battle: The Agent Avalanche
---
[00:21:24]
[00:21:26] Phil: B-b-b-boss battle: the agent avalanche.
[00:21:32] The boss on this floor is the agent avalanche. It's like a torrential downpour of agents operating on default assumptions, the implicit belief that someone else made the governance decisions, that the vendor's interface is just a UX choice, that coordination happens automatically once agents are running.
[00:21:50] It shows up as the governance gap, the space between what agents can do and what they should do, and as the default interface, the vendor chat box that [00:22:00] quietly became your dispatch layer without anyone really deciding it. Defeating both versions of this boss requires making each decision deliberately before the consequences make the decision for you.
[00:22:14] All right. Let's check our inventory to see how we battle the agent avalanche. In our inventory, we've got an agent governance framework.
[00:22:25] We also have the observability stack to continuous anomaly detection across every data source and agent output. We also have deterministic layer, how we handle dates, math, and field selections so the LLM never has to guess. And the last one is a dispatch team: data engineers, privacy specialists, and traffic cops who govern routing across the entire marketing org.
[00:22:51] Let's put these items to work
[00:22:53] Why You Need a Central Referee, With Agent Guardrails
---
[00:22:53]
[00:23:01] Phil: Guardrails are an organizational question before a technical one, and they should belong in the architecture from the start. The right frame is what AI should do in each context, scoped before the agent runs
[00:23:13] Tiankai: So whatever you're launching, you have the right guard rails on it. If it's dealing with personal information, make sure it's not linkable or there cannot be any bad things happening to the data, right?
[00:23:23] That they're dealing with.
[00:23:24] Phil: That's Tianyi Feng, data and AI strategy director at ThoughtWorks and the author of Humanizing Data Strategy.
[00:23:31] Tiankai: But it's actually much more a service design problem or like a user experience problem even, right? Because if you think about a user coming and they has to face 10 different AI agents, all of a sudden, which should be maybe one seamless interaction with you as a customer, then you are, again, back to what I said before, with 10 campaigns in one week, you are overwhelmed, right?
[00:23:51] So it should be actually much more driven by is that the right thing for our business and is that actually helping our customers to be [00:24:00] loyal? Buyers of our product or services, then thinking how can we launch more AI agents, right? Um, if we rooted back to what is helping us to be successful in a business and not just what is exciting to do and what is quick to launch, I.
[00:24:14] Then I think the whole mindset can shift more towards end-to-end thinking, value thinking, design thinking, all the things to actually make it better.
[00:24:22] Phil: The practical version of what Sianke's talking about in principle is governance drills, running scenarios where something breaks in production.
[00:24:30] Chris: you have to be really clear about governance.
[00:24:32] You have to be clear about when are you going to let agents run autonomously and where are you not?
[00:24:37] Phil: That's Chris O'Neill, who's a now board member at GrowthLoop and who helped contribute to the 2025 AI and Marketing Performance Index.
[00:24:46] He borrows this concept from cybersecurity.
[00:24:48] Chris: and companies need to be really clear about, about who, like data strategy, who owns what. Um, a big fan of red team drills, right? So you actually simulate, uh, uh, hallucination, [00:25:00] uh, IP leakage, like being really clear about governance of data, who has rights, who basically is responsible, basically makes sure you track and then understand what data is being used for, what like that needs to be treated. Think of like, uh, you know, asset management, like this laptop I have at a company.
[00:25:17] Like we track, like assets like that, you know, historically haven't done that with data, which is arguably one of the most important assets any company has. So. Similar to the red team drill I mentioned before, you need to simulate to make sure that your practices hold up in an emergency or that if something goes wrong, you know who, who is responsible for what, when there's IP leakage, you know, when there's hallucination in something core, um, I think you have to have a mindset.
[00:25:44] That data really matters, and then you have to have a mindset that there's going to be stuff that goes wrong. This is frontier technology, and I think the teams that, uh, are succeeding have realistic expectations. They basically don't think it's gonna solve everything and not have [00:26:00] hallucinations or have failures.
[00:26:00] It's going to, and then just get over it and have processes in place to, so to, to know who's gonna be responsible for what, when this thing. It doesn't work the way you expect it to. That's just the nature of new technology. So maybe that's how I think about the data piece and, and how it relates to governance and red team drills and like really being thoughtful and, uh, really nimble, um, with, uh, with the underlying data itself.
[00:26:24] Phil: The teams that do this are distinguishable from the ones that don't. You can call them out specifically because governance drills force clarity about authority. Who can shut off the agent? Who gets paged when the targeting logic behaves unexpectedly? Who owns the dataset that the agent read from? In 2026, like every one of those is an operational requirement when stuff hits the fan.
[00:26:48] Sponsor: Knak
---
[00:26:48]
[00:27:55] Sponsor: MoEngage
---
[00:27:55]
[00:28:52] Privacy Compliance and Data Minimization
---
[00:28:52] Phil: The 2026 State of MarTech report from Scott and friends found that marketers are pursuing an average of 70 [00:29:00] distinct AI use cases across their orgs. That number sounds like, you know, really positive progress, but it also describes a governance issue. Most of those 70 use cases are being stood up faster than the foundational infrastructure that would've made them safe to run on in the first place.
[00:29:17] Things like data lineage, uh, compliance, privacy, consent, they're not advancing at the same rate as the more visible, more celebrated use cases at the top of the list. That's not good. The glamorous work like AI-generated copy, autonomous audiences, predictive lead scoring, they're all moving really, really fast.
[00:29:36] The unglamorous work, knowing what data the agent is using, whether it had permission, compliance, data, bad records, like that moves really slow and if it's moving at all. And every AI initiative eventually runs into the same question: Do we actually trust the data, context, and permissions this agent is acting on?
[00:29:56] The teams that answer that question before they need to are the ones [00:30:00] who don't find out the answer from a customer complaint
[00:30:03] So privacy and compliance are where this stops being this like abstract governance conversation. If an agent can read customer data, generate campaign copy, enrich a profile, suppress an audience, or whatever, it's now operating inside the same risk surface as the rest of your MarTech stack. You need to enforce permissions, consent awareness, lineage, approval paths, and a clear answer to who owns the decision when the agent acts.
[00:30:31] This is where privacy becomes architecture, and consent can't live in a PDF or a chat box somewhere or policy doc that an agent never reads. It has to be operational context
[00:30:44] Michele Nieberding: Like there's so much that goes into consent management. But if you're not enforcing that consent as soon as the data is collected, that's a problem,
[00:30:53] Phil: That's Michelle Nieberding, who's now head of product marketing at Optiversal, made this point in our [00:31:00] episode on customer data and infrastructure on server-side data processing.
[00:31:04] Michele Nieberding: Right? Because then what happens is you're collecting data. You're not sure that those Um consent preferences are actually being enforced. So then your data goes downstream and you're like, oh crap I've just identified a compliance risk like all it's there's there's a problem by the time you identify it like It is a problem, you are in triage, like, you could be potentially fined, all these things.
[00:31:27] there are lists and lists of third party data breaches, so, you know, sites going down due to third parties, data breaches due to third parties, like, third parties causing so much Pain and so many problems that our goal is to give companies control back of their customer data and eliminate that reliance on third parties.
[00:31:45] Um, which is really a fun challenge to, to take on. So
[00:31:49] Phil: Michelle also makes the same point from the preventative side
[00:31:53] Michele Nieberding: you don't want to wait until it's problem to do something about it. Like, you know, the floodgates are [00:32:00] going to open and you don't want to be the one at the, you know, bottom of the dam.
[00:32:02] Like, oh, no. Um, and I say that because when I say first party, I want to be clear there's like first party data collection. So you're asking people for their personal information. You use that for targeting, right?
[00:32:13] That's one thing. can be really hard. You still have to deal with PII, all those things, right? When you build a platform and you have a platform within a first party infrastructure that goes to first party endpoints, Um, You're really collecting more information that's like behavioral data interactions events on your website So you're collecting more and better data, but you're doing so in a way that Strengthens your resilience against browser, you know ad blockers things like that I kind of like to say like don't let browsers boss you around because it's a real thing And so the way that we collect it is getting you more and better data that you can feed into all your systems.
[00:32:52] So your CDPs, your analytics, your data warehouses, wherever you might send it, because it's within that first priority party endpoint, [00:33:00] um, which is really exciting because like, again, being a tech nerd at heart, you don't. Realize what your data can do until you realize like what good data, like the, the impact good data makes like we hear all the time, maybe bad or like not full sets of data going into a CDP, for example, like maybe anonymous, anonymous user data.
[00:33:21] You can't identify those users. That's not going into your CDP and you're like, oh my CDP is, you know, doing all these great things. Yeah, but you're missing Half your data because you're not tracking anonymous users, right? Like The struggle is real.
[00:33:33] Phil: One thing that's worth mentioning here is the uncomfortable tension of context versus data minimization. AI systems get better when they have more context, but privacy-first marketing depends on restraint and minimizing how much of that data you're keeping. So there's an interesting paradox there. The right context is not all of the context.
[00:33:54] It is the minimum necessary context for the decision the agent is allowed to make. That means [00:34:00] consent, data minimization, privacy rules have to become part of the context layer itself versus this, like, separate review step after the campaign is already built.
[00:34:10] Siobhan Solberg: And this is why I always say that the data protection bit needs to be intertwined.
[00:34:15] Phil: That's Siobhan Solberg, privacy data and AI governance advisor
[00:34:19] Siobhan Solberg: The data strategy obviously includes also where you're storing it, how you're maintaining it, um, how you're collecting it, how you're processing it, how long will you keep it for.
[00:34:30] And some of these questions that I just said are actually driven by data protection. Meaning, how long am I keeping the data for? How long am I legally supposed to keep the data for? And how long can I keep it for? What's the reason I'm collecting it? These are all little questions now that you have to ask within your data strategy.
[00:34:47] That are really influenced by data protection that we didn't ask beforehand. Maybe we should have. But we didn't, Hehehe.
[00:34:56] so it kind of all plays together. And I think the one thing I will always [00:35:00] say, sit down with the various teams within your company as soon as possible. Don't wait until you've built your great strategy and then bring in legal or the compliance officer or whoever it might be.
[00:35:11] It's too late. Bring them in in the beginning because they can have that conversation for you with you. And at the same time, they might have really great ideas on how to actually make that happen.
[00:35:21] Phil: She also gives the data minimization side of that simple rule
[00:35:25] Siobhan Solberg: Yeah, I think that there, we need to be careful, right? Because just. In case data is not always just in case data and just in time data is usually too late. So there is a balance that needs to be found I feel that we got to get really lazy because we had all this data, just, you know, granted it was probably a mess, but we had it all.
[00:35:47] Um, and, and we could just never have to think about the strategy or what we might need to do with it or where we're going to be going with it and how this all fits within the context of the business. Now, I think we need to be a lot more strategic and we [00:36:00] really need to think ahead.
[00:36:01] Phil: All right, so even a well-governed and compliant stack breaks. The question is whether you find out before your users do and complain, and then your CEO finds out and blah, blah, blah.
[00:36:12] Kevin Hu: when it comes to data quality, the question is not whether data quality issues occur, right? They will occur.
[00:36:21] Phil: That's Kevin Hu, CEO of Metaplane, was recently acquired by Datadog.
[00:36:25] He describes the design objective.
[00:36:27] Kevin Hu: The question is, do you want to know about it before your users do? And do you want to get better at it over time? Um, the faster we kind of, you know, to say yes to both of those, like now we're starting to build up some muscle and some core competency.
[00:36:44] Phil: Data quality is a continuous monitoring problem. We covered that in episode two of the series. And because every new source, every new transformation, every new tool in the stack opens another door, and the teams that build observability as a system are the ones who stop finding out about [00:37:00] broken data from the customer or the CEO.
[00:37:03] Liz: we really needed Marge to kind of, again, with the vision of you have a 24 7 analyst who understands your business. She need to understand the disciplines of marketing.
[00:37:12] Elizabeth Dobbs built observability directly into the agent layer at Databricks as a structural feature of how Merge, one of their marketing agents, operates. she, we don't like release her to the wild and we're like, good luck everyone. Like she, we do look at, she has a feedback loop. She had thumbs up, thumbs down. We do benchmarking, we kind of look at all of the data on a very regular basis.
[00:37:35] I think we spend about like a no, an hour or two a week going through all the feedback and making sure if we're getting bad answers, how do we improve her? So it's not just like a, an agent that's out there by itself. It is constantly kind of governed, reviewed, and we, um, well, you know, if we have issues or we will reach out to the marketer and say, Hey, we can see your whole query.
[00:37:55] Why did you go like this? What, what, what felt wrong? And just really make sure we're constantly improving her. So I [00:38:00] think those three things are kind of, uh, you know, what we've been doing to make sure that Marge is tru, trustworthy, actionable, and really kind of able to keep pace with the marketers here.
[00:38:09] Phil: So governance and observability. Governance here makes velocity sustainable for the agents, and observability makes governance a, a thing. Like, you, you can't govern stuff that you can't really see. But there's still a question underneath both of those two. Observability tells you when something breaks.
[00:38:27] The dispatch layer decides what was supposed to happen in the first place, and there's a layer above both of those that most architecture discussions aren't really thinking about right now
[00:38:38] The Dispatch Layer that Controls Routing Logic
---
[00:38:38]
[00:38:46] Phil: Most of the conversations around AI orchestration focus on what happens behind the interface, like which agents are running, how they hand work to each other, what the coordination layer looks like. But most [00:39:00] organizations are thinking about orchestration and are completely skipping about the question of who built the door.
[00:39:07] Who built the door? What does that even mean? Let me expand on that. So every marketer on your team opens some interface every morning, a UI, right? Like a chat box, a command bar, a conversational AI feature in one of your tools. They type what they want, something happens, maybe they're clicking buttons here and there.
[00:39:26] But between the question and the answer is this like routing agent, a master agent that decides which tools and which sub-agents and which workflows actually get invoked. Whoever controls that routing agent controls what gets used downstream.
[00:39:45] Aboli: you know, as you're thinking about creating agents or connecting them to each other, there is the agent orchestration.
[00:39:51] So how do these, you know, agents talk to each other?
[00:39:54] Phil: That's Aboli Gongridawar, general manager of lifetime value at Credible and someone who has been [00:40:00] building agentic infrastructure for marketing operations. She talks about what it takes to turn individual agents into something that can actually run end to end.
[00:40:08] Aboli: if I were to take a lifecycle marketing use case, uh, if I'm sending out an email campaign, right, like I could have a email copy agent, an email HTML coding agent, right? how does my copy agent intake.
[00:40:22] Copy. You know, like if there's a Figma agent that build, builds out the email or like visualizes the email or designs the email and then, you know, there's the coding agent, then that quotes out the email. Like right now, I think where, um, where a lot of teams are are, you know, they're probably building out these agents, you know, individually, uh, like testing around it.
[00:40:43] But then you need an orchestration layer eventually for these agents to sort of pass the copy to Figma from Figma to the HTML billing. Interface, right? So that it's all like autonomously done. And, um, so I think at some point, right, that orchestration layer needs to exist.
[00:40:58] Phil: So individual [00:41:00] agents are proof of concepts, but orchestrated agents that hand work to each other seem a lot more like a production system
[00:41:07] Rebecca: I think there's gonna be a new team forming, um, a new, uh, part of this organization that's gonna expand from marketing ops and MarTech.
[00:41:17] Phil: That's Rebecca Corliss. She's the VP of marketing at Leapsome, but previous to that, she spent two years leading marketing at GrowthLoop. She calls this team the dispatch layer, a new hub between marketing leadership and execution pods
[00:41:31] Rebecca: So imagine now this new desk dispatch layer, that is the group that's thinking about the systems, the data, the ai, the architecture. And campaign activation for the entire marketing marketing org holistically. It's really interesting. You think about it, and the reason why we call it dispatch affectionately is because we think this group is going to be accountable for how all marketing communications will go out in a way that's really effective for all of [00:42:00] the marketing objectives that are being fulfilled.
[00:42:01] Phil: So the dispatch team is the right answer to orchestration.
[00:42:05] For most teams, it starts as a single person with documented sets of standing decisions like which agents are active, what the suppression rules are across overlapping campaigns, and who gets paged when something breaks. Whatever the title though, like someone needs to also own the routing logic explicitly, uh, and also maintain a written governance doc and run quarterly reviews on, you know, which agents are running and why.
[00:42:29] Rebecca: So there's likely someone who's thinking about security. There's likely someone thinking about, uh, I don't know what we'll call the role, but the traffic cop, we'll go with traffic cop who is thinking about, like, I'll give you example in this pod world, I bet we'll need to be really sophisticated around what campaign.
[00:42:49] Is each customer receiving which business unit is serving this particular customer? And when this customer could actually effectively, uh, be a [00:43:00] part of any business unit's campaign, who decides that? I think that's gonna exist at the dispatch layer based off of what serves this customer best. Uh, this customer should actually be part of this campaign and this outcome.
[00:43:11] So interesting to think about. Um. Others privacy. I think there's gonna be, uh, the data individual is gonna be really crucial, likely multiple. I think some of the coordination that's gonna happen between the pods and the dispatch layer is going to be, okay, what goals are you looking to achieve? That's clear.
[00:43:28] What data do you need to achieve that goal? What do you not know about your customer now? And how can we invest in getting that understanding in order to serve the customer best and, uh, serve your campaign your outcome best? So that's gonna be an important role too.
[00:43:43] Phil: I feel like that's the minimum viable version of this function, and obviously as the team grows, complexity kinda grows and demands with it. But there's a layer above the dispatch team that I wanna talk about here that a lot of organizations haven't thought about yet. That's the interface [00:44:00] itself.
[00:44:00] The Layer Above the Dispatch is the Interface
---
[00:44:00]
[00:44:07] Phil: I'm a big fan of the Foreign Key newsletter written by Florian Delval, who's at Snowflake, and he wrote about this piece in, uh, published in April 2026. Behind every conversational interface, every chat box a marketer types into, there's a routing agent that decide what's gets, what gets invoked. Whoever controls that agent controls what gets used downstream.
[00:44:30] So he says, quote, like, "If you don't control the interface, it doesn't matter how much orchestration you own. Users may never reach your agentic network. It's like building the most advanced city in the world, but controlling none of the roads that lead to it." He's also got a really cool grocery store analogy in there that's, um, pretty instructive here.
[00:44:50] Like the store is a commodity interface, unremarkable and interchangeable, but the store employees decide which products are visible, which [00:45:00] ones get surfaced, which ones get picked. The interface is a commodity in this sense, but it controls the outcome. D2C brands spent like the last decade trying to bypass the retailers specifically because of this dynamic, but in the agentic marketing stack, the same logic applies.
[00:45:16] The vendor whose chat interface your marketers opens every morning is functionally your dispatch layer, whether you intended that to be the case or not. Florian says, "In theory, nobody should care too much about the interface.
[00:45:31] In practice, it determines control."
[00:45:33] Lindsay: So at Zapier, we've got access to agent... several agent harnesses that we can, um, sort of decide based on preference.
[00:45:42] Phil: Lindsay Roethlisberger lives this from the inside at Zapier. The AI center of excellence that she's part of is functionally the routing layer, so to speak, for the organization, the team that decides what agent harnesses get provisioned, what context gets connected, and what governance applies before [00:46:00] anyone sits down to use it.
[00:46:01] Lindsay: So a lot of folks are primarily using Cowork. Um, some folks are, like, living and breathing in Cursor or Claude Code.
[00:46:09] So it's, we leave that up to the end user.
[00:46:11] But what we did do is this, uh, center of excellence stood up out of the go was a golden path to Cursor.
[00:46:17] It's really step-by-step instructions for non-technical folks, um, in setting up their Cursor environment, with instructions how to connect Databricks MCP, how to connect Zapier MCP, all the rules and guidelines around safe use.
[00:46:33] so yeah, centralized management, but then the hub and spoke where I'm sort of, like, on the ground in go-to-market, and then the shared brain being, like, a major part of building that initial, uh, rollout out.
[00:46:44] Phil: So whether Cursor connects to Databricks MCP or Zapier MCP, what context loads by default, what rules govern safe use, like all of that was decided before the user sat down. The interface feels like autonomy, but the [00:47:00] defaults are the architecture in this case. Teams that build their own conversational layer, like Elizabeth Dobbs at Databricks with Marge, they're retaining that routing control.
[00:47:10] The teams that built to a vendor's AI interface, they're basically handing it over. The orchestration you built is still there, but the marketer just may never actually reach it
[00:47:20] What Production AI Agents Look Like on a Governed Data Foundation
---
[00:47:20]
[00:47:28] Phil: So I wanted to close out this section or come close to close it out by talking about, um, some of the production AI agents and, and what those look like on a governed data foundation. So at Databricks, Elizabeth Dobbs, uh, and her team built three production agents sitting entirely on top of their marketing lakehouse.
[00:47:47] So we've got Marge. She handles conversational data access. She's trained on specific vocabulary and definitions of every marketing discipline in the org. Tagatha automates content tagging [00:48:00] continuously and retroactively. And Atlas handles segmentation, combining rule-based and intent-based logic so audiences can be defined once and updated when new ones are surfaced.
[00:48:12] Every agent reads from the same governed source with no copies and no sync jobs
[00:48:17] Liz: we're building up is this kind of, uh, idea of the one chat. So you have like a chat interface very similar to like the one text bar, Chad, GB like thing. But on the backend we just kind of have all of our genie who are trained with, hey, what their specializations are. So the idea is the marketer doesn't really know that there are all of these genies out there.
[00:48:36] There's all these genie rooms that they have to navigate between them because that's just. Hard for, I think someone to keep in their head all the time is, Hey, I asked a question. Did I go to the right gen room? Is this the right answer? Do I be like, so we are trying to obfuscate all the complexity in it, but the idea is yes, if a marketer has like a 1 0 1 question, Marge is the place that will will take them.
[00:48:57] But if they're going to go deep, they're trying to really get, [00:49:00] um, you know, a deep research answer, which has been really helpful for our team, which is like kind of the, the way you can actually have Marge do kind of deep analysis on insights. That's where we really make sure you have to end up in the right kind of genie room for all additional context, much more narrow scope so she can be much more effective in in her answers.
[00:49:20] Phil: Obviously, this is super cool stuff. Like the routing, specialization, the orchestration are all absorbed by the architecture itself, and the marketer is just asking questions, and, and the two other ones are kind of operating behind the scenes. Um, yeah, the, the 2026 State of MarTech report, to call this back, like for the 17th time in this series, it dec- describes the vendor stakes really interestingly here.
[00:49:44] Like in the last era, MarTech platforms competed to be where marketers worked, the place people logged into every morning. But in the next era, they're competing to be what agents can work with. The race has moved from desktop real estate to [00:50:00] MCP integration, API surface area, and agent readiness, if you will
[00:50:05] Platforms that position themselves as substrates for agents rather than the destinations for humans are making a different kind of bet about where value gets captured next. That is the SaaS to substrate shift, and it's already determining which vendors are building towards the center of your stack and which are quietly being routed around it
[00:50:27] The Deterministic Layer: Why the Interface Can't Run on Guesses
---
[00:50:27]
[00:50:34] Phil: All right, let's chat about the deterministic layer here, because controlling the interface is only half the problem. The other half is what happens when the interface actually tries to answer a question. Ed Campbell is the CEO of Bright Analytics, and he spent months building an MCP server for marketing analytics, documented every way it goes wrong.
[00:50:56] Uh, he wrote about it on LinkedIn here. I'll link this out. Really cool [00:51:00] post. The first instinct of pointing an LLM at your raw data and starting asking questions to it kind of works enough to be exciting, not enough to trust it. Campaign in Google and ads and campaign in Meta are separate concepts and separate tables.
[00:51:17] The AI picks whichever spend field looks more relevant, and it won't flag the ambiguity. Like it returns a number with complete confidence, but the number might actually be wrong. The underlying issue here is a structural one, because an LLM is probabilistic. It guesses confidently and with reasonable calibration, but it guesses.
[00:51:37] It's probabilistic in nature, right? Dates, math, field selection, those things can be guesses. Like last quarter might mean a calendar quarter or a fiscal one. Um, year to date might start in April for a business. A business whose working week runs Friday to Thursday have last week that is nobody's else's last week.
[00:51:59] Like [00:52:00] without a deterministic layer handling these questions, the analyst has to re-explain their calendar every single time. It's like a super expensive way to go about some pretty manual work. So Campbell's team built a semantic layer between the raw data and the MCP, a structured layer that encodes domain knowledge once permanently, and the LLM only ever sees the metrics your team has explicitly defined.
[00:52:26] So we're using some of the stuff that we built in, uh, episode two, part two of this series, the semantic layer context stuff. And things like dates get a dedicated API that resolves against each other, uh, each customer's specific calendar. There's like arithmetic things that get pre-calculated on data objects 'cause LLMs can't really do reliable math.
[00:52:50] Um, so the LLM in this case is interpreting intent and deterministic systems are executing it for it. Um, my favorite quote in his post is, [00:53:00] "Neglect the architecture behind your MCP and you have a very expensive, very confident liability. Get it right and you will have something transformational."
[00:53:09] System Stewardship: The Role That Emerges When You Clear This Floor
---
[00:53:09]
[00:53:15] Phil: All right. We are almost through our boss battle here on the floor. Um, let's talk about system stewardship before we close things off.
[00:53:25] Anna: what I always talk, talk about is don't be inhibited by what the AI tool can offer you, your imagination. Is the limit, right?
[00:53:34] Phil: I had a really cool episode with Anna Abashian, who at the time of recording was, uh, VP of operations at Civic Technologies. She rebuilt her entire analytics stack around this principle, routing warehouse data into an LLM client so any stakeholder can ask questions in plain language
[00:53:51] Anna: So we're shifting from being execution focused to now being very design focused. And I think that that is happening across many different [00:54:00] organization. You are now, you know, taking your ops team and creating, um, an an environment where you're not driven by tasks, but you're driven by like, how do I architect solutions now?
[00:54:12] How do I design workflows that really help? You know, myself, like to self scale and then also my team and the organization, uh, the organization to scale better. Um, so I, I, I love like, you know, being able to, as we talked about like AI tools have democratized like building, I mean, how many of us have like, vibe coded on the side, like doing things that were just like unimaginable before, right?
[00:54:37] Like The sky is the limit. Um, just think about ultimately what your objective is and then design from there. Um, AI tools, they, the landscape transforms so quickly at breakneck speed.
[00:54:50] Phil: And let's hear from Matthew Castino, marketing measurement science lead at Canva, who's describing what the data layer looks like when it's mature enough to stop bottlenecking the [00:55:00] people who need it
[00:55:00] Matthew: working a lot with Snowflake and the Cortex product to try and introduce.
[00:55:04] Phil: Hmm.
[00:55:04] Matthew: Natural language querying of the data warehouse, um, for stakeholders. So what we're trying to focus on, data science is creating a semantic layer in our warehouse, which has, you know, the core context that the model, that the model needs to answer questions in a reliable fashion. And then we're working towards a world where, so there's, you know, there's low hanging questions that sort of like, um, probably exist in a dashboard somewhere that someone can't find or.
[00:55:34] Um, that don't take that long to answer individually, but when you get heaps of them, they take up heaps of time. Um, trying to create this like natural language interface for our stakeholders, um, so that they can, um, you know, answer questions or get answers to questions quickly, um, without requiring a data scientist, allowing a data scientist to focus on more complex work like developing experimentation, [00:56:00] tooling, so on and so forth.
[00:56:01] Phil: Boom. Let's wrap up with what we've sprinkled in throughout this entire dungeon crawl here. But, you know, the 2026 State of MarTech report from Scott and friends describes what is happening to the roles on every team that clears this floor. We've got campaign managers that are becoming agent operators, then value engineers, people whose job shifts from executing campaigns to defining what good outcomes look like and ensuring agents pursue them.
[00:56:29] And we have system administrators that are becoming stack wranglers, the context engineers, the people whose job shifts from maintaining tools to designing the shared context layer those tools and, and agents are reading from. And the report uses the, uh, chrysalis metaphor, right? Like a butterfly is a different thing assembled from the same material, built during a period that looked from the outside like nothing was really happening.
[00:56:55] So it's a really cool way to kinda coin this floor together here. Like, teams [00:57:00] that use this transition to rebuild their job around higher leverage work are gonna run this next area. And, you know, it's crazy times for sure, but teams that wait for it to settle first are, you know, gonna find the roles already defined without them.
[00:57:16] Uh, let's, let's close things out with, uh, Elizabeth Dobbs, who is, is running a bunch of agents at, at Databricks
[00:57:23] Liz: And I think agents are of really great heuristic to be thinking about because. I actually do think a lot of agents are not yet ready to be fully human outta the loop, fully, like prime time on their own, but let's say in 12, 18 months, like they will absolutely be there. there And so kind of what our job is, I think as a, a marketing technology team is kind of just to place really strategic bets like you're in Vegas, like.
[00:57:47] All right. It, we might be hedging here, but we really feel like the industry, the velocity, what we know about the space is gonna catch up. And we don't wanna be the team that's like, been waiting for it to be perfect and then be ready to start. [00:58:00] Um, so I think, you know, you, you have to think about, I think the idea of like the one-way door, right?
[00:58:04] Like what are the one-way door decisions where we're gonna go through and it's really hard and painful to step back. And what are the things where we think we can make, uh, architectural decisions, which, you know, could be painful up front, but will pay us dividends in the long term.
[00:58:20] Phil: I love it. Like takeaway there for me is build towards human out of the loop from where you actually are. But, uh, let's call it back and, and close things off with David Chan's sentence, "Alignment, not integration, will be the ultimate rate limiter." We spent two decades learning to connect systems to each other.
[00:58:40] The pipes are largely solved. The next two decades are the semantic design problem, shared definitions, causal foundations, institutional memory, and the governance that keeps humans in the loop without bottlenecking the agents that act on their behalves. The question running through every conversation in this series is the same [00:59:00] one that surfaces over and over.
[00:59:02] When every system has its own version of the customer, its own definitions, assumptions, local truth, how does a company decide what's real and what's counterfeit? Solving the alignment problem requires a clearer theory of what the data is actually for. It looks like a team that can answer three questions really well. What did the agents do last week? Why did they do it? What would change if we wanted them to do something different next week? If your team can answer all three, you've cleared the dungeon.
[00:59:33] FINAL ACHIEVEMENT: The Dungeon Is Cleared
---
[00:59:33]
[00:59:33] Phil: Final achievement, the dungeon is cleared.
[00:59:37]
[00:59:43] Phil: Four bosses, four floors all cleared. The false truth king in the CRM, the export hydra spreading it everywhere, the hallucination oracle spewing believable nonsense from agents, the correlation boomerang archer scaling the wrong behavior at speed, and finally, the [01:00:00] agent avalanche operating with default assumptions, the quiet belief that someone was already made the governance decisions.
[01:00:07] What you have on the other side here is foundation that can learn, audiences that belong to no single tool, and agents that interpret data with the right context, a causal record that grows more reliable with every experiment you run, and a dispatch layer where the governance decisions were made on purpose by a person with a name and a document.
[01:00:29] This dungeon was super fun to put together. Obviously, But our work as MarTech crawlers is never truly done. The vendors will keep sending emails. The bosses will respond in new forms. Teams that cleared these four floors will at least recognize the patterns next time it shows up because they've already seen how each one hides.
[01:00:49] The dungeon crawl advantage for marketers here is pattern recognition. You learn to read it before it reads you. Thank you folks, if you're still hanging around, for descending with me, and [01:01:00] huge thanks to all of our guests for the past 20 months that joined me on the podcast and contributed to this episode.
[01:01:06] We'll catch you next
[01:01:08]