Subscribe
Share
Share
Embed
AI agents are being positioned as the answer to shrinking budgets and rising expectations, but most enterprise teams are still stuck in pilot mode.
In this Office Hours episode, Jacqueline is joined by guest host and industry analyst Keanu Taylor to examine what's actually working inside large organizations. Drawing on insights from his research, the conversation explores why many "AI strategies" amount to fragmented experiments, and what it really takes to move from internal pilots to external decisioning. Instead of chasing one all-powerful agent, leading teams are breaking work into hierarchies of specialized micro-agents, backed by better data context and governance.
ROI shows up in unexpected ways: efficiency gains often unlock deeper strategic insight rather than just cost savings. And none of it works without adoption, which means real hand-holding, expectation management, and treating data governance as infrastructure, not a meeting-room exercise.
Timestamps
00:04 - Introducing Keanu Taylor and the marketing technology shift
01:42 - What 13 enterprise consumer brands are testing with AI agents
02:56 - Navigating the era of doing more with less in martech
07:10 - Why autonomy doesn't mean AGI: the rise of micro-agent hierarchies
10:33 - AI decisioning agents explained and how they're finally delivering value
15:02 - Two-sided ROI: efficiency gains that unlock effectiveness and insight
20:52 - Adoption reality: mistrust, user inertia, and the need for hand-holding
31:21 - Moving data governance from meeting rooms to infrastructure
Sponsor
Brought to you by Hightouch - Went all-in on a big marketing suite but still struggling to get value? You're not alone. Our sponsor, Hightouch, spoke with 50+ enterprise teams and found 79% are frustrated by high costs, slow innovation, and rising complexity, often needing specialized teams just to keep things running. They'll share the full findings in a live webinar on February 12, plus what they're seeing from organizations updating their Martech stacks. Get the report and register!
Connect & Subscribe
Creators and Guests
What is Making Sense of Martech?
Unfiltered takes on the biggest shifts in marketing technology. We spotlight what matters, who's leading (or lagging), and what's next. In Martech, clarity is power — and we're here to deliver it.
00;00;04;28 - 00;00;20;27
Speaker 1
Welcome to the Making Sense of MarTech podcast. I'm Jaclyn Friedman, and today we have a guest host, Keanu Taylor, for office hours. And we're cutting through the AI noise, which is a little different than what we normally do. But we want to understand where real value is emerging and enterprise marketing agents are being touted as the next big thing.
00;00;20;27 - 00;00;39;21
Speaker 1
But what's hype? What's actually driving real business outcomes? First off, I want to introduce you to Keanu if you don't already know him. He leads industry analysis of the most important shifts in marketing technology, working directly with hundreds of brands to understand and support how they choose, use and abuse martech within the organizations. We're gonna have to coin that one.
00;00;39;21 - 00;00;42;02
Speaker 1
Keanu. It's a.
00;00;42;02 - 00;00;47;20
Speaker 2
Favorite of mine, choosing as many as a lot of brands out there. Definitely abusing some of our tech.
00;00;47;23 - 00;01;06;02
Speaker 1
That's very true. Well, we've got a special treat ahead because he has a brand new report that is a comprehensive look at a genetic AI adoption across enterprises. He's charting use cases from internal copilots to decisioning engines and exploring the data, governance and psychological hurdles along the way. Welcome.
00;01;06;04 - 00;01;08;05
Speaker 2
Thank you. Thanks for having me.
00;01;08;07 - 00;01;24;27
Speaker 1
Of course. All right, let's get down to the meat. We don't want to talk about any hype. We only want to talk about the reality of things. So your report really opens up with this concept of do more with less. And if we're going to enter in this era, how did this context shape your research into AI agents?
00;01;25;00 - 00;01;50;08
Speaker 2
So so maybe before I get into the context, so the research report, we or I sat down with like 13 enterprise, consumer brands and, talked to them about AI agents, how they're adopting AI agents. What, has been working, what has not been working, what use cases they've been, chasing down. And I think the context became clear anecdotally.
00;01;50;08 - 00;02;08;13
Speaker 2
So people are being asked to do more with less. This is that I've to do more with less era. It's a bit of a cliche in enterprise organizations. Everyone's been told to do more with less. You've got less budget. You've got less, less people. But just get on with it and do better than you did last year.
00;02;08;14 - 00;02;33;11
Speaker 2
I think there's been this shift. So obviously in the Covid era, we had all of this investment in digital transformation and marketing technology and marketing more generally got, oh, they saw a lot of that investment. There was a lot of brands shifting to more e-commerce models or doing more, more online for their brand, more digital. And then after a while, those budgets, suddenly disappeared.
00;02;33;12 - 00;02;56;20
Speaker 2
Everyone was getting back outside again, and everyone was saying, okay, great. We don't have to invest. In the same way we were. Obviously, the interest rates went through the roof. Macroeconomics became a lot more challenging. And suddenly, brand budgets were being tightened quite a lot. Marketing and martech, particularly, wore, the brunt of a lot of that.
00;02;56;22 - 00;03;17;13
Speaker 2
Which is sort of led us to this era of doing more with less. But at the same time, there was this, white knight that appeared, which was I, we've got this amazing technology. It'll make things so much easier. You'll be able to save time, money, etc., etc.. We know that a lot of that, in the early days was, was hype.
00;03;17;13 - 00;03;40;15
Speaker 2
It was excitement, folks. Really excited about the possibilities. But now the rubber is really hitting the road. People, you know, or brands, rolling out proofs of concept, pilots actually, stress testing, you know, testing it in anger. And we're now at this position where it's a bit of a bit of a turning point. It's like, is this going to give us the the returns?
00;03;40;15 - 00;04;03;27
Speaker 2
Is it going to make things easier to deliver in marketing? It's a really exciting time to actually be going and talking to these brands about how they're adopting AI agents. But interestingly, through through all that, the investment in AI in the last three years is tripled. According to some research from the marketing AI, Institute. So there's this dichotomy of, budgets are getting smaller, but more of it is going to AI.
00;04;03;28 - 00;04;09;20
Speaker 2
And with the promise that it will help people through the do more with less period that we're in at the moment.
00;04;09;21 - 00;04;12;27
Speaker 1
But of course, I can solve everything.
00;04;12;29 - 00;04;14;14
Speaker 2
Well, well, yeah.
00;04;14;16 - 00;04;15;25
Speaker 1
You, Well, but also.
00;04;16;00 - 00;04;35;03
Speaker 2
If you ask Sam Altman, then, yes, it will it will take 95% of, marketing jobs in the next three years or something ridiculous. But I think through, through the research we've done, we found it's a lot more nuanced than that, of course. There are there are valuable use cases. There are use cases that are struggling to to show that value.
00;04;35;06 - 00;04;41;01
Speaker 2
But folks are surely certainly not dying wondering on AI agents. There's a lot of investment going into it.
00;04;41;04 - 00;05;03;09
Speaker 1
Speaking of that investment, it feels like a complete contradiction that executive priorities are allowing investment in AI to be through the roof. Meanwhile, just your standard marketing and martech budgets are tightening. What do you foresee from the exact level why they're saying we have a strategy. That's an AI strategy because we have the budget for it. And what are they actually meaning when they're saying this?
00;05;03;09 - 00;05;06;28
Speaker 1
Can you please translate and like do the bullshit detection?
00;05;07;00 - 00;05;25;05
Speaker 2
I mean, a lot of the time we have an AI strategy means we have a bunch of separate pilots that are not very joined up, across different teams that are not talking to each other. I'm being unfair on some companies. There's definitely companies where, you know, there's great executive buy in. There's joint up strategy across teams.
00;05;25;06 - 00;05;48;24
Speaker 2
But for a lot of companies it's very fragmented. The understanding and use of AI agents, in their business. I spoke to the head of martech at a health insurance company. He said that his, is executive said tells everyone. Yep, we have an AI agent strategy. And then, he, of course, looks around and says, oh, that's interesting.
00;05;48;24 - 00;05;52;27
Speaker 2
I wonder, I wonder who's leading that? And, apparently, apparently it's him, so.
00;05;52;27 - 00;05;57;20
Speaker 1
Well, that's a surprise promotion question mark.
00;05;57;22 - 00;06;21;26
Speaker 2
Yeah, it's, maybe, maybe a poisoned chalice, but, yeah, there's always executive pressure. Of course, it's a lot of excitement, but I think that's always a double edged sword. I mean, on the one hand, yes, pressure and expectations is increasing, but on the other of these companies now have a proper mandate, for sustained investment, being able to take people away from other jobs and put them towards this.
00;06;21;26 - 00;06;53;14
Speaker 2
So I think it's, it can be a positive. You have to harness it in the right way. And sure that executive expectations, managed, you know, don't under-promise. It's a classic marketing technology, you know, under-promise and overdeliver. That's, that's the way to to live. But of course, you know, all these executives, a hearing in their ear to hearing crazy things about AGI and all this nonsense when there's actually a lot of value to be, to be, gained without any AGI or anything like that being, involved, for sure.
00;06;53;15 - 00;07;10;29
Speaker 1
So we've talked about what's happening across the board from a budget perspective, but I want to know what's actually happening, what is moving the needle. And when we think through that, like what is the most common misunderstood banding we have about the autonomy AI agents can bring to the workforce?
00;07;10;29 - 00;07;37;13
Speaker 2
I think some of it comes from this AGI narrative. So the idea that agents will do everything for you start to finish, it's very reductive and not really that helpful. I think through the research that we we've done, we found that in a lot of instances, these brands are actually trying to break up big problems into a lot of smaller problems and create smaller agents or micro agents to solve each of those problems in sequence.
00;07;37;19 - 00;08;05;22
Speaker 2
I think this idea that one agent rules the model, another thing we found is brands are building hierarchies of agents. So like a like an org design, you know, there's there's one agent that, sort of orchestrates a few others. I think that's that's where it's heading in terms of autonomy. It's almost like this is operating model of agents rather than just having a, I don't know, like, I'm trying to think of a movie where, where there's like an AI assistant, this this is where I struggle.
00;08;05;22 - 00;08;08;19
Speaker 2
I don't watch enough movies. Show. Chill. You've got.
00;08;08;22 - 00;08;10;01
Speaker 1
Her. Maybe.
00;08;10;01 - 00;08;28;03
Speaker 2
Yes. Yeah. Yeah, her. So there's not there's not one, you know, computer out there that that does it all. It really is, you know, these big problems divide up into smaller, easier to solve problems and then and then orchestrated. I think that's one of the things it's like autonomy. You don't want to give one agent autonomy to do everything.
00;08;28;03 - 00;08;46;26
Speaker 2
That's where trouble comes. Breaking it down into smaller things. Building governance at every single layer. So, you know, it's a lot harder to manage one agent that has access to all your data and can do anything versus having, you know, let's say there's an agent that can only see customer data and that's its specialty. And then you can build governance around that.
00;08;46;27 - 00;08;57;05
Speaker 2
And, infrastructure around that. I think that's probably the biggest misnomer or the biggest misunderstanding. But, you know, there's there's plenty of other things that that folks are struggling with for sure.
00;08;57;05 - 00;09;06;00
Speaker 1
And we've been alluding to there's different types of agents. You outline a couple different, definitions. Would you like to share them with us?
00;09;06;06 - 00;09;34;05
Speaker 2
Yeah. So I guess the the big difference here is, is internal and external. I think that's the nice clear delineation, especially on the brand side. So of course outside of the brands control, you've got the ChatGPT phase of this world where, you know, brands have less control. Of course there's geo and everything, but for the purpose of this report, we really looked at, total agents which help, people doing their job and doing their workflows.
00;09;34;05 - 00;10;00;18
Speaker 2
And when I say people, I mean, you know, proper human people. And then there's external facing agents, which have a role in interfacing with customers with a degree of autonomy. So, you know, decisioning agents, content personalization agents, there's there's two different levels of, I think, risk or at least perceived risk brands that we we spoke to for this piece of research, a lot of them saw internal agents as being a lot lower risk.
00;10;00;18 - 00;10;23;17
Speaker 2
Of course, it doesn't immediately present things to customers, so of course there's a lower risk there. But a lot of them, are using internal use cases to build towards an external use case. So they're trying to prove the concept internally in a safer environment and then try and chase down the external and customer facing or consumer facing use case.
00;10;23;19 - 00;10;32;28
Speaker 1
That makes sense. And thinking of external specifically, are there any of these external agents that are past the hype phase and actually proven?
00;10;33;00 - 00;10;58;10
Speaker 2
I think so, I think so, the one that I'm most interested in and saw a lot of value in is, decisioning agents and it's interesting because we talk about agents as this umbrella term for actually a lot of different AI and machine learning techniques under the hood. So for decisioning agent, it's more about reinforcement learning. You're talking about, multi-armed bandits, sort of under the hood.
00;10;58;10 - 00;11;24;17
Speaker 2
But I guess the crazy thing about this is actually being possible for a long period of time. It just hasn't been productized. So when I when I started my career, we actually built a machine learning or, reinforcement learning. You know, we built it from scratch, model, which could as say, did, like repurchase communications from automotive, manufacturer and it could constantly change which or change and learn which Colin to suggest to the person.
00;11;24;17 - 00;11;48;23
Speaker 2
So suggest the top three change subject lines. It could change content images, or all of this stuff. But we built it from scratch. It was very, very manual. It was not scalable at all. It, fell over a few times, but that was that was 13 years ago. But now it has been productized. There's a lot of, AI decisioning, you know, decisioning agent type, products on the market.
00;11;48;23 - 00;12;11;09
Speaker 2
Now, I think there's a lot of value here. Of course. You know, this is a customer facing application of AI agents so that there is there is more risk factor. But I heard from a few different people. I think everyone is coalescing around this idea of the perfect use case, particularly for external facing agents, is, something that is high volume.
00;12;11;12 - 00;12;33;19
Speaker 2
So you've got a lot of customers or people that you can impact medium complexity. So high complexity, maybe the AI is not there yet. Low complexity can. It's more, deterministic in nature. So it's just not necessary to apply, to that. So medium complexity and the last one is low risk. So that's what these martech ladies are looking for in terms of use cases.
00;12;33;21 - 00;12;49;19
Speaker 2
Find an audience. That's that's not going to be that risky. So for for some brands it's like the prospect audience. So they don't own the products that are actually have the relationship. If you mess up with that, with that customer, with that prospect, you're not at risk of churn. They've never ends.
00;12;49;26 - 00;12;51;08
Speaker 1
Just that risk for brand.
00;12;51;08 - 00;13;00;09
Speaker 2
But of course, there's, it's the brand risk. But, this is where there's a little bit of, a bit of bravery required from people.
00;13;00;12 - 00;13;00;20
Speaker 1
That.
00;13;00;20 - 00;13;20;03
Speaker 2
Know there's so many controls. And I think, you know, the companies that the product ties in in this space have built a lot of controls and guardrails. And yeah, it's interesting. I think a lot of people like the idea of AI where you're like, download an app, switch it on, and then it goes and does all the hard work.
00;13;20;03 - 00;13;28;25
Speaker 2
But the hard work is actually moving from defining how, you communicate with customers to defining how not to communicate with customers.
00;13;28;28 - 00;13;30;16
Speaker 1
It's an important exercise.
00;13;30;18 - 00;13;47;23
Speaker 2
Yeah, yeah. So instead of saying, okay, well, you communicate with customers in this way, in this journey, this step follows this step. If they do this do why what markets now have to do is say, okay, this is this is how you don't communicate with these customers. Don't do this, don't do that. Yeah. And then you end up with a set of guardrails.
00;13;47;25 - 00;14;02;00
Speaker 2
All the acceptable treatments, sort of within the guardrails. So, yeah, I think I think certainly decisioning, is one of these applications where there is a lot of values. We had a lot of companies, a lot of different industries that that could benefit from it.
00;14;02;02 - 00;14;22;26
Speaker 1
I love that we're basically hearing folks dogfood being like, as a gateway drug to confirm business cases so that we can do externally, which is interesting because the easiest use cases oftentimes are the external based on what you're sharing. But you have to prove it first. And so speaking of proof, I would love to talk about ROI because this stuff is not cheap.
00;14;22;28 - 00;14;31;10
Speaker 1
And so I'm curious you discuss this two sided ROI. So both efficiency and effectiveness. Can you give an example of a use case where both sides showed up?
00;14;31;15 - 00;15;02;09
Speaker 2
Yeah. So I think it's interesting with with AI agents, a lot of the narrative has been around efficiency and doing the same things, but quicker, cheaper, you know, more easily. But there is a in some ways and a lot of the use cases doing something more efficiently actually, drives impact in the longer term. So for example, I spoke to a bank based in the US and they basically built out like an analytical agent or a talk with your data type agent.
00;15;02;09 - 00;15;31;05
Speaker 2
So folks in the in the business could type in a natural language query. It would go and build, you know, SQL in the background. You know, present charts give numbers and original objective of that project was to defer questions from the analytical team or the analytics team. So, you know, these highly paid, very, sophisticated analytics folks were getting you've worked in these business before, you know, the questions that get a lot of the time, they're repeated.
00;15;31;05 - 00;15;33;29
Speaker 2
Someone's already asked the question, but I didn't communicate it.
00;15;33;29 - 00;15;38;23
Speaker 1
And I'd even say they don't even know what question they're asking. I'm just the.
00;15;38;23 - 00;16;08;11
Speaker 2
Type. And that as well. 100%. Yeah, yeah. There's a there's a level of knowledge that's required, but they wanted to defer those questions, or at least the simple questions so that the, the analytical team could focus on things that, you know, I can't answer, you know, deeper exploration of their data. But very quickly they realized that business users who, you know, not not, technically minded, they can't write SQL to to find out information.
00;16;08;13 - 00;16;30;21
Speaker 2
They were asking more and more searching questions, and some of them are actually finding insights that they had not previously found which have impact on their business. So although it was all about efficiency, let's, let's, free up our analytics team very quickly. They realize, oh, there's actually an impact here. You know, these people are finding insights which they otherwise would have not found.
00;16;30;21 - 00;16;44;17
Speaker 2
They would have been completely buried or, you know, they're buried in the backlog somewhere. So I think there's this two sided ROI where efficiency actually begets effectiveness and impact. In a lot of cases. But it's a it's a longer term play for sure.
00;16;44;18 - 00;17;09;08
Speaker 1
Sounds like music to my ears. And a quick word from our sponsor, has your company gone all in on a large marketing suite but struggled to implement or see value from the investment? You're not alone. Our sponsor, High Touch, analyzed conversations with over 50 enterprise teams on large marketing suites and found that 79% report frustrations with high costs, low innovation, growing complexity, and often requiring specialized teams just to operate them.
00;17;09;08 - 00;17;36;12
Speaker 1
They're sharing those findings in a live webinar on February 12th, where they'll walk us through the patterns are seeing and how some organizations are slowly updating their martech settings. If this is something you've been thinking about, join the live session to get the full report. Register today and high touch.com/suite fatigue. And now back to the episode. And so if you're putting yourself in the shoes of a brand side martech person when we're building this business case, are we talking hours save revenue gains or something else.
00;17;36;12 - 00;17;44;20
Speaker 1
Like what are maybe some of the best ways to think about it? Because this was an unexpected finding that folks are at least experiencing right now.
00;17;44;22 - 00;18;11;19
Speaker 2
I think the AI agenda is actually so broad at the moment. There's so many use cases. So there's there's a lot of different metrics and, measures of success. And, you know, being used in these business cases. I think on on the efficiency side, this is probably one of the challenges because especially for marketers, the market is coming from this world where a lot of their business cases are built on, growth and, you know, revenue growth.
00;18;11;21 - 00;18;37;02
Speaker 2
Can I convert more customers? Can I push more customers through or consumers through the pipeline, you know, through the marketing funnel. And as marketers and martech leaders are not always naturals at, doing the efficiency, sort of metrics and measuring how much time is saved, how many people is it saved across? So I think one of the one of the big learnings was you got to get very sophisticated at measuring those things.
00;18;37;02 - 00;18;51;20
Speaker 2
And, you know, a lot of the time it's hard. You ask, you know, in that last example with the analytics team, they ask how many, how many queries a week did you get? How often, you know, what's the average time it takes to respond to one of those queries? You know, some of those.
00;18;51;20 - 00;18;53;08
Speaker 1
It's hard to get those metrics.
00;18;53;11 - 00;19;14;02
Speaker 2
Of course. Of course. And some of those, but some of those numbers, it's like a finger, finger in the sky, you know, a wet finger in the sky. But I think the, the teams that are doing really well and building really strong business cases are able to numerator and quantify, the benefit and I guess the also the benefit of freeing this person up.
00;19;14;05 - 00;19;29;28
Speaker 2
So, you know, a lot of brands are really focused on can I get my people doing more strategic, deeper thinking, deeper work. It's definitely a focus for, for the brands, moving people from, from low value work to high value work.
00;19;30;02 - 00;19;53;10
Speaker 1
I love the sound of that. And I'm sure some execs to and based on all the conversations as execs expectations just keep being a huge theme, not just for agents, but really just managing expectations is hard. Yeah, and I'm curious, based off all of the people you spoke with, how are martech leaders handling this pressure, particularly as it relates to this?
00;19;53;12 - 00;20;18;06
Speaker 2
This is one of those spaces where it's not really that different to any other tech trend in the past. I think the principles still apply. Even if you look at pays, you know, 6 or 7 years ago and the the excitement and hype, and there's a lot of brands that didn't manage expectations well there. I think the main part of managing expectations is the time and investment.
00;20;18;06 - 00;20;28;06
Speaker 2
Just being really clear to say the the investment will be bigger than we think it is. It will take longer than we think it will take.
00;20;28;08 - 00;20;29;26
Speaker 1
Always add some buffer.
00;20;29;29 - 00;20;52;04
Speaker 2
Exactly, exactly. There's yeah, there's a few cases where, companies got into, you know, creating some sort of AI agent and then they get a month in and realize, oh, we actually have to build a context layer for our whole business. And then they go, okay, well, that's an undertaking in itself. And there's all of these infrastructure pieces that people don't realize.
00;20;52;07 - 00;21;10;03
Speaker 2
So I think the message I would certainly give to executives is to say, this is what we want to do. This is what we know. There are gaps here. There are things we don't know which are going to pop up. It's hard to predict sometimes, you know, for another company, it was it was redoing their tagging, redoing how they tag content.
00;21;10;03 - 00;21;29;08
Speaker 2
They had to do it three times. And that's the time. Oh yeah. Yeah. That's the type of project delay and added investment, which is always painful to executives. And I hate it. I hate surprises, but I to say that there will be surprises. I think managing expectations to say we will need sustained investment. This is what I think it is.
00;21;29;08 - 00;21;36;26
Speaker 2
I am sure there will be other things that crop up. I know that's, not always. There's definitely executives who wouldn't accept that message, but.
00;21;36;26 - 00;21;39;19
Speaker 1
Yeah, it's a hard one to deliver, that's for sure.
00;21;39;21 - 00;22;09;04
Speaker 2
But I think this is this, like, you wrap it around the idea of this being quite speculative in a way. You know, the the gains are uncertain. A lot of brands, achieving sort of you know, some real gains here, but the gains are in certain. So, positioning as such. Again, it comes back to the technology in, in the days of, you know, very deterministic technology, it's quite easy to quantify what the uplift in what the benefit is going to be when it comes to more probabilistic technology.
00;22;09;07 - 00;22;28;00
Speaker 2
You really just don't know until you try. So it's a bit of a switch in mindset from, you know, we know exactly what we're going to deploy, we know what the challenges will be, and we know what the cost will be. Even though a lot of companies have been terrible at doing that in the first instance. But, now it's, you know, we know it as a, as a range that it will fall in.
00;22;28;00 - 00;22;38;16
Speaker 2
We know that that's probably going to be some things, some infrastructural things that crop up that make it difficult. But yet that's some of the challenge with, managing expectations of the executive at this point.
00;22;38;19 - 00;22;58;25
Speaker 1
Yeah for sure. And while it's probably hard to not just manage the manage the executive, it's also kind of hard to manage the team because in your report, you also touch on a psychological friction. So of course there's fear of replacement, mistrust of AI and just general user inertia. And I know adoption takes a long time and it takes a lot of enabling it.
00;22;58;28 - 00;23;04;15
Speaker 1
And I guess what were some of the strategies you saw that were successful to overcome these types of gaps?
00;23;04;17 - 00;23;27;06
Speaker 2
Yeah, I think it's reasonable that employees and these big businesses maybe a bit mistrustful, a bit concerned. So much of the narrative is around job displacement. And it's interesting because most of that narrative comes from big tech. You know, the Microsoft, Google, Salesforce is these these are the companies that, laying off people and saying explicitly it's for AI.
00;23;27;07 - 00;23;45;01
Speaker 2
Through our research, I mean, it's only, certain companies that we spoke to, but not a single one, spoke about displacement. Most of them talked about actually, more investment and more people to manage and build, these technologies. So there is this narrative, which can of course be a bit of a concern to folks.
00;23;45;08 - 00;24;09;24
Speaker 2
And then, the other one that you touched on is the mistrust and AI, as I said, it's probabilistic. For the most part, people need a change of mindset to understand how it works and understand that it doesn't follow nice, neat, predictable rules, as a lot of marketing technology has in the past. But in terms of what what have companies been doing to overcome this?
00;24;09;24 - 00;24;49;19
Speaker 2
I think the first or the most important one, and this is not a deeply technical, strategy, but handholding. I think that was the word that came up time and again. Folks, folks need hand-holding. They need to be assured that this is about, building extra capacity and not about taking away, jobs, the hand-holding, you know, there's there's longer periods of, like, hyper care with this technology that is taking time to really get folks so are going to use it to understand the underlying data, where, where things are coming from, what decisions are being made over which, which data points.
00;24;49;21 - 00;24;54;15
Speaker 2
So, yeah, the operative word was was hand-holding, or at least that's the word I heard a few times.
00;24;54;18 - 00;25;28;15
Speaker 1
Yeah, I'm not surprised. And also I'm going to read between the lines in terms of the more macro trends that are concerning for the regular day employee, the big name companies that are laying off a lot of people, they're having to invest in something that they don't know how to invest in. And so I think actually the narrative is they're using those headlines to pretend like they know what they're doing and cover up some of their own mismanagement and misplaced responsibilities on a budgetary and planning standpoint.
00;25;28;15 - 00;25;29;29
Speaker 1
But that's my own hot take.
00;25;30;01 - 00;25;47;07
Speaker 2
Yeah, I think that's a that's a pretty solid hot take. I mean, a lot of these it comes back to the era that we're in post-Covid. You know, a lot of these big tech companies they hired massively. And I think they're still adjusting from that. I'm not sure how much of this is truly about AI, but at the same time, right.
00;25;47;10 - 00;26;05;04
Speaker 2
Some of these companies, they're actually hiring. So, you know, Klarna has been in the news. They got rid of 607 hundred of their customer service stuff, and then nine months later they had to hire them back because customer service, you know, see, SAT and satisfaction went through the floor. So, you know, there's there's a.
00;26;05;04 - 00;26;06;00
Speaker 1
Lot of cries.
00;26;06;02 - 00;26;19;17
Speaker 2
Yeah. There's a lot of a lot of narrative, a lot of unnecessary hype and nonsense out there. But yeah, I think for, for serious brands at the moment, it doesn't really seem like there's much, much, displacement or desire to displace going on.
00;26;19;21 - 00;26;44;28
Speaker 1
Yeah. Agreed. All right. We've alluded to different technical components of the stack, but I really want to dive in. You've kind of mentioned this concept of hierarchy of micro agents more than just building one super agent. Could you explain maybe for everyone who's trying to visualize this, like the concept of a united data layer, a unified semantic layer, like what matters, what's going to be helpful?
00;26;45;00 - 00;26;53;17
Speaker 1
And granted, this is a, you know, high level review of it, but I would love to hear how you define and describe that design.
00;26;53;21 - 00;27;14;07
Speaker 2
Yeah, I think unified data layer is absolutely key. It's been key for a long time. Brands are spending a lot of money trying to invest in it. But I agents and I more broadly is it's another one of these sort of triggers for, for more investment in bringing data together. So that one data layer came up time and again, it's absolutely critical.
00;27;14;07 - 00;27;37;05
Speaker 2
But also this idea of the semantic layer or the context layer or whatever you want to call it, companies need to give these agents a lot of context to to help them to succeed. And the challenge is a lot of context. Traditionally in business sits up in people's heads. And of course there's this knowledge basis. There's, you know, there's information on the website, it's document management.
00;27;37;05 - 00;28;10;17
Speaker 2
But at the end of the day, there's, sort of tribal knowledge. There's just experience that sits in people's heads. So part of the challenge is how do you get that information written down, how to get it written in a way that is consumable, to AI and some of the really cool things I saw were companies that using AI to help solve that problem so they would actually get AI to, you know, collect a data point or comment from someone and iterate on it and create, I see, like natural language sentences about that piece of context.
00;28;10;23 - 00;28;37;11
Speaker 2
And one person I we saw a use case. They called it a context tree. So every, every single data point about a customer and then booking with the company, would be written out using generative AI, written out in natural language sentences. So, you know, Keanu Taylor has a booking at, 1230 on this date, for this vehicle or whatever was, was relevant information.
00;28;37;11 - 00;29;03;06
Speaker 2
So the unified context layer is this, additional layer above the data layer. So more context, more semantic information and then the hierarchy of agents. So I think everyone so comes into AI agents or a lot of people come into, adopting AI agents thinking there will be one agent will deploy one agent, and it will do a few things.
00;29;03;07 - 00;29;06;05
Speaker 1
One to rule us all. That's the case.
00;29;06;07 - 00;29;32;14
Speaker 2
Lord of the rings style. They thought it would be a one ring, but it really hasn't proven to be the case and it may change in the future. But at the moment, the best way to create predictable and and probable sort of winning guardrails results is, breaking down these big problems into smaller problems, creating smaller or micro agents to just solve that problem or be an expert in that space.
00;29;32;17 - 00;29;56;20
Speaker 2
So again, that this bank I spoke to, they I created the business agent, which was the sort of top level agent. And then under the business agent, they had, an agent for loans. They had an agent for withdrawals. They had an agent for, products and sort of like a product expert agent. And the business agent would orchestrate between each to decide who was best placed to answer the question.
00;29;56;22 - 00;30;17;27
Speaker 2
And sometimes it would decide there's actually multiple of the agents that are required to answer the question comprehensively. There's sort of starting to be built in, in hierarchies, and it's interesting. It's sort of resembles a an old structure, but are basically only full of subject matter experts. Right. So you can have an agent for, you know, orchestration.
00;30;17;27 - 00;30;30;04
Speaker 2
You can have an agent for, a particular product, agent for a particular table or, you know, database within your organization. That's sort of where, where companies have been going. I see that's where they see the value. See it work.
00;30;30;05 - 00;30;40;01
Speaker 1
It's almost like the the deep secret model is actually what's working in practice, at least within companies. Yeah, potentially.
00;30;40;03 - 00;31;01;26
Speaker 2
I think so I think so this hierarchy of agents was, was definitely, an interesting finding through this. And I think brands have honestly just stumbled upon it. I think slowly the narrative is changing and people, you know, tech companies are actually talking about, you know, these hierarchies or, agent orchestration. But it was selling for me that was that was one of the big learnings from from the, the research.
00;31;01;28 - 00;31;21;16
Speaker 1
Without a doubt. And we've kind of alluded to it in some capacities. But something really important we haven't discussed is governance, especially with all of these highly regulated companies that you've spoken with and I'm curious, you you write about governance as infrastructure. And how are folks actually operationalizing that?
00;31;21;18 - 00;31;45;22
Speaker 2
Yeah, traditionally companies, a lot of their governance exists in meeting rooms, basically. You have governance councils, you have meetings where you talk about governance and you talk about challenges and what's coming up and, you know, making decisions and decision frameworks and rescue models and a lot of governance layers. Again, in, in people's heads, in, meeting rooms, in people's boats, in their notebooks.
00;31;45;22 - 00;32;20;00
Speaker 2
But increasingly, as AI agents scale, governance cannot be done by humans. It's just it's a nonstarter. Otherwise, it sort of defeats the purpose of of using AI agents in the first place, which is to scale your efficiency impact. So more of the burden for governance, is moving into technical infrastructure, so into the platforms themselves. And to be fair, I think vendors in the space are doing a really good job now of recognizing that need and building out very minor use controls within a better.
00;32;20;02 - 00;32;47;05
Speaker 2
I mean, they won't sell their product if they don't, especially to some of these regulated brands, regulated industries. But the the level of control and this is also part of the reason for hierarchies of agents, I think, which is instead of having one agent where you try and describe everything it can and can't do, you have smaller agents, talking about a specific subject matter or a specific table or specific part of the business, and they have very tight controls on them.
00;32;47;07 - 00;33;06;24
Speaker 2
And then the, the sort of overarching or orchestrating agent it can sort of refer to or defer to each, agent, ask a question, are we talking about loans? Right. I'll go to the loan agent, I'll ask a question. And that agent has all of the knowledge. It has all the controls, all the guardrails, built into its infrastructure.
00;33;06;25 - 00;33;27;29
Speaker 2
So, yeah, there's there's this shift of governance away from, right from people to technical infrastructure. I think for for IT teams, it's probably a, you know, a big challenge that they're facing. And, you know, martech teams as well. But having that level of governance that's baked in otherwise, again, it's it's very difficult to scale this in an appropriate way without making mistakes.
00;33;27;29 - 00;33;53;13
Speaker 1
Without a doubt. All right. Let's close this conversation out with a few things everyone can act on today. So it sounds like the easiest tip is to start with an internal low risk use case that drives productivity TBD on how you measure it. There's a lot of options. Second, don't skip the change management. Train your people as seriously as you can, figure the tools or just hold their hand the whole time.
00;33;53;15 - 00;34;11;20
Speaker 1
And lastly, really invest in data readiness. The context you need for your agents to actually deliver value is so much more important than anything else. If you don't have that, you basically have to start again. And it sounds like we've heard of a couple of companies. I had to start over a couple of times in order to make it possible.
00;34;11;23 - 00;34;24;02
Speaker 1
Yeah. Well, on that note, thank you so much, Keanu. You can find his full report. AI agents in the enterprise from pilot to proof at the MarTech weekly. Thanks for listening and see you next time on Making Sense of Tech podcast.