Subscribe
Share
Share
Embed
Serious People Podcast: An Operator's Manual to AI is a podcast for business leaders and operators running complex, real-world businesses. The ones selling supplements, managing caregivers, or running service crews. Not software.
Host Noah Levin brings nearly two decades of experience at Amazon, Whole Foods, and healthcare tech to weekly conversations about what AI outcomes work inside a business, and what doesn't.
Equal parts practical and irreverent, but useful above all else.
[00:00:00] Christina Johnson: Look at your sales data. The number of people who don't actually look at their sales data along with their marketing efficiency data is insane. Like you can run a really efficient campaign that drives no sales. The example I have for that is that someone told me... They ran this, super efficient marketing campaign in, like, LA, New York, big cities.
And you know why they didn't drive any Walmart sales? 'Cause there aren't Walmarts in those cities.
So it looked great from a marketing metric perspective, but the second they looked at the sales, that's a pretty big whiff. And that's where, you need to look at your actual business metrics alongside your marketing metrics.
[00:00:34] Noah Levin: Welcome to Serious People. I'm Noah Levin. Today, my guest is Christina Johnson, founder and CEO of Advisar, an intelligence tool for retail media buyers. If your business uses performance marketing, and what business doesn't these days, you're gonna love this conversation. We get into why analytics are so hard to get right, what happens when you get it wrong, and how to use tools like Claude to make better decisions with the data you have.
So if you or someone you love is spending gobs of money on ads, this one's for you.
[00:01:07] Noah Levin: So we know each other from business school, from Michigan. How did you end up there?
[00:01:10] Christina Johnson: It's a windy story. So, like, I moved to LA. I was like, "I wanna work in film and TV." Managed to get introduced to the producers of Grey's Anatomy, like, right when they sold the idea to Grey's Anatomy to the network. We were developing the pilot. I was, like, the junior person. I was basically, like, photocopying scripts and getting M&M's, like, not doing anything fancy.
But, like, got to kind of be on this ride where Grey's was nothing, and all of a sudden it was, like, nationally famous. Which was interesting, but I just realized I wanted to have a life, and you just cannot have a life, in the film and TV industry in the way that, I wanted to be able to, exercise and not always be stressed out.
I was always on 24/7. So tried a couple other things, like non-profity. Wound up at all places, in the executive office at Goldman Sachs doing also, like, junior-type things,
[00:01:58] Noah Levin: Wait, wait, wait, wait. How do you-- Sorry. How do you go from getting coffee on the set of Grey's Anatomy to the Goldman Sachs office?
[00:02:08] Christina Johnson: Yeah. So that was like the 2008, 2009 downturn in New York. And so like I'd been in LA and I was like, "Okay, it's time to move back to New York, to my family, is." And I was just like looking for jobs, and it was really hard to find a job. And so like submitted my resume to this placement agency who loved the Hollywood background and was like, "Oh, you'll do well on Wall Street 'cause you're kind of unflappable with like famous people."
And so like interviewed in and they were like, yeah, got placed on that floor and it was right when the CEO of Goldman was like being interviewed by the SEC. It was like peak like Occupy Wall Street era. But I was like, "I'm, I mean, I'll do anything for a story, really, Noah."
So did it turn out to be true that having worked in Hollywood on the set of Grey's Anatomy, you were, like, well-equipped to work on Wall Street?
I would say, like, yeah, the skills translate, and I would say the skills translated to Walmart, too.
Getting good at, like, observing and not needing to insert myself into a situation and listening has, served me really well from all of those situations.
[00:03:08] Noah Levin: I wanna, fast-forward and get us to the point where you are at Walmart
[00:03:13] Christina Johnson: Yep
[00:03:13] Noah Levin: and finding yourself learning about retail media and how wildly backwards it is
[00:03:19] Christina Johnson: So basically at a high level, you know, and Amazon was the first mover on this, like retailers started to realize that they were sitting on huge piles of data, and that they could use this data to target shoppers across their platform. So like, the example I always give is like Walmart knows so much about its shoppers that an advertiser, like say I have a toothpaste I'm releasing and I wanna be able to market to moms, with kids age 5 to 12, with my new toothpaste.
Like I can go to Walmart and say, "Hey, I wanna buy the ability to target that mom when she is searching on your site for toothpaste. Like you know, you know all that about her. I want my ad to come up when she searches toothpaste or when she searches something else kid adjacent." Up to like that point, it was really, really hard for marketers to prove out like value, like they call it closed loop attribution.
It was really hard for marketers to be like, "Well, I ran these ads," and then I can close the loop and say, "It definitely drove these sales." And on retailer sites, because the shopping is happening on the site, you can theoretically, and there's a lot of controversy around this, but you can theoretically say, "Hey, you ran that ad," and then someone clicked through the ad and they bought the thing.
So the ad drove the sale. Now, did the ad actually drive the sale? I'm not gonna go down that rabbit hole, but it was sort of a new, a new way of being able to measure advertising. Retailers realized how valuable this was, because when they sell the advertising like that, it's not like selling something that's a product.
You get like 50 to 90% margin selling advertising, and suddenly you can prop up other parts of your business and do all sorts of fun things with those profits. I was working for someone whose job it was to grow the advertising business, and I understood like, even as a marketer, like I understood so little of it that I was like, "I must be an idiot."
Like, 'cause people are talking about things and like I don't know what they're saying, and it sounds crazy to me. And
[00:05:13] Noah Levin: What, what's an, what's an example of something that they were saying that didn't make any sense?
[00:05:18] Christina Johnson: Like, it was just all these random metrics. They would just talk numbers at you. And I'd come from this marketing of, like, you build a strategy and, like, who are you targeting? And, like, there are numbers at the end, but these people were just throwing out, like, the fill rate was this, and the click-through rate was this, and the this.
Like, it, it felt like gobbledygook.
And so that was when people were, like, really didn't know what was going on. So, like, I was not stupid. I was just listening to other people's also trying to figure it out and thinking they knew more than me when really we were just all, clueless together.
[00:05:50] Noah Levin: Tell me what your job was at Walmart when you were working on retail media
[00:05:53] Christina Johnson: So I was, the lead strategist and then the chief of staff to Janey Whiteside, the chief customer officer.
Our team was tasked with things like figuring out who the design target should be for Walmart at that time, like who Walmart should be going after, how should we be talking to them, what are the services they want. We were in the front of the board, or she was in front of the board all the time, because we were building up things like, Walmart Connect and, like, Walmart Plus, which is Walmart's membership, which were kind of newer additions.
So spent a lot of time kind of incubating new businesses for Walmart. That was my job.
[00:06:25] Noah Levin: So you're, inserted into this business that has a mandate, it has traction, it doesn't have scale, but it has sort of this inevitability of “We know it can get there, we are Walmart.” Were you operating this business? What was your role?
[00:06:36] Christina Johnson: So like there was a time at Walmart, and this happens at almost every company that starts a retail media network that does not come from a digital background, is like you have to get the leadership of the company bought in on doing a media company.
Walmart at the time was like a 50-plus-year-old brick and mortar retailer that had just kind of started doing e-com seriously. And so a lot of the work that like my boss did that we were doing was convincing the C-suite that, the sales team for Connect did in fact need to be incentivized by sales targets and budgets, which was just new at Walmart at the time.
So it was like figuring out like what the org structure needed to be like, figuring, the convincing the C-suite that those changes needed to happen.
One of the big things was like you can't run a successful retail media network without buy-in from the rest of the organization. So like how, what does that look like in terms of KPIs for the merchants? So I was doing a lot of that higher level work.
[00:07:36] Noah Levin: Tell me at what point retail media kind of became like your thing. And when you first found yourself in a role where you were, actually like in the sh*t with retailers trying to explain to them why they should spend their money or why their money was spent well or not spent well
[00:07:56] Christina Johnson: It kind of fell in my lap after Janey ended up leaving, I worked for the, uh, head of Walmart Connect. Um, and then Janey actually asked me to, to work with her at a boutique consulting firm. And it just so happened that like retail media was just, exploding at the time. And so we ended up getting a whole bunch of work consulting for other like Fortune 500 companies that were building up retail media networks.
And it was, how do you build out a sales team? How do you get organizational buy-in? What are you offering suppliers? How are you selling into them? It was like setting all these fundamental things.
It's funny that like everybody assumes everybody else has it all together. They'll be like, "Oh, well Walmart knows what they're doing. Oh, well so-and-so knows what they're doing." You're like, everybody is, maybe strategically guessing, but, it's like saying they. Who is they? None of they have it figured out
[00:08:42] Noah Levin: You mentioned a specific meeting when you were consulting where it sounded like no one knew what was going on. Would you want to tell us about that meeting?
[00:08:50] Christina Johnson: I was consulting for a retail media network and we were talking with some of the sales team and we were just like, "Well, how are you communicating to suppliers, like what they're getting?" "What do you mean? We're like showing them the placement and maybe we talk about impressions."
And we're like, "But like sales lift or like..." And it just became really clear that the sales team themselves didn't know how to look at the data in a way to make an argument to suppliers that, their spend had worked, that they should be spending more. And that was somewhat shocking to me because this was a retail media network that was like selling hundreds of millions of dollars in ads.
So it meant that there was suppliers on the other side just also taking this word that like things are gonna work.
And so it was just shocking to me to see the people doing the selling like, A, not having the data or the information to be able to justify to convince people to sell more, but then that the suppliers on the other end weren't asking those questions.
[00:09:44] Noah Levin: Why weren't they asking? What is it about this, format that just they weren't pushing hard
[00:09:49] Christina Johnson: I... I think there's variation in sophistication in understanding marketing because retail media has suppliers who are not historically, marketing organizations needing to spend marketing dollars. There can be some organizational pressure where they feel like if they're selling at a retailer, then they need to spend on their retail media network in order to get visibility.
So it's a new muscle for a lot of people. So some people I think just didn't know or thought they were just servicing the the, the function. And then I think others, like I was saying, I was totally confused when I first started in retail media. I was like, "I don't know what any of this means." I think people don't wanna look stupid, and so they don't ask the question.
[00:10:23] Noah Levin: If I wanted to be a smart retail marketer, go spend a pile of cash but spend it efficiently on my toothpaste brand,
What are the questions I should be asking? What are the metrics I should be looking at?
[00:10:33] Christina Johnson: I mean, there's like internal questions, right, that you need to be asking, and that's for like the people who are not used to marketing. Like, what are the goals of this campaign? Are you actually trying to just drive sales, or are you trying to drive awareness? 'Cause if you're trying to drive sales like your sell through and your unit velocity, are the things you're gonna look at as KPIs.
But if it's awareness, like you are gonna look at things like reach and frequency. Your new to buyer percentage, so you're like bringing in new people to your category. So getting clear on that internally on your own side is good. And then asking a lot of pointed questions to the retailer, like,
What are the incremental sales that came from this campaign? Like not the sales that would've happened anyway that you're claiming credit for, but like, what sales can you tell me above and beyond, were driven by this campaign? And that's a really hard thing to answer.
I don't envy the retail media networks. It's not simple, even though it sounds so.
[00:11:20] Noah Levin: Tell me when, like, it goes horribly wrong? Is there an example of where, someone just blasts the money cannon in the wrong direction and has nothing to show for it?
[00:11:31] Christina Johnson: It can go horribly wrong a couple of different ways. Like, so more spend doesn't actually necessarily mean better marketing. Marketing is more nuanced than I think people think it is. Like you need to get surgical about like who you're trying to target and then like what tactics you're going to target.
So if like, and this happens all the time where people are like, "Oh my, I'm being told to just spend more." And you're like, "But that doesn't actually do anything."
[00:11:54] Noah Levin: Well, yeah, you t- you mentioned there was a supplier that, had to use auto-bid or was being told to use
[00:11:58] Christina Johnson: Oh, yeah. There's a function in retail media where you can just like basically set it and forget it.
It's not actually really that effective. They were being told by the network to do that. They actually ended up spending $0 and moved zero items because they weren't being picked up by the algorithm. So it just like fell flat, like even though they were being told it was the right thing.
And so we ended up doing a more manual campaign, which was more work, and then they ended up selling through their whole pallet of items. So it just goes to show like there's a lot of incorrect information out there, and it's really easy to just misspend or pick the wrong thing. But also like it's okay to do that and fix it. Like marketing is forgiving.
[00:12:32] Ad Read: This episode is sponsored by Valet. Christina and I talked about how much easier it's become to use AI to analyze your data. Once you've built something useful, you need a way to share it. Valet lets you publish the dashboards, analyses, websites, and small tools you build with AI at a stable URL. You can keep them private for your team or make them public.
We used it to make a companion page for this episode with three lessons from Christina, prompts you can try with your own campaign data, and a worksheet to organize your review. You can find that page in the show notes. And to publish something of your own, go to valet.dev in your agent today.
[00:13:13] Noah Levin: Today, you're the founder of Advisar, which is an AI-driven product that helps marketers understand their spend in retail, right?
What got you to the point of realizing that AI was a tool you could deploy against this messiness and, worldwide ignorance?
[00:13:31] Christina Johnson: When I was a hands-on marketer, I used to have to cobble together all sorts of disparate sources. I'd have to pull like massive spreadsheets to get to any sort of answer about whether something worked or not.
[00:13:41] Noah Levin: Where were you pulling data from?
[00:13:43] Christina Johnson: I mean, everything from like Walmart's internal point of sale data to, reach and frequency information for TV ads to digital metrics like click-through rate, to things like, Nielsen data. It really was just like disparate and it was so dependent on like whether you happened to think of the thing at the time.
I can't even imagine being a marketer with like just Claude right now, even at its base. Like you used to just have to like kind of run around and ask people for information and like pray that they gave it to you. And it was very happenstance, but I learned to get pretty smart with cobbling together things that would give me directionally correct information.
You could see that the campaign worked because people were adding the product we were advertising to their basket at a higher rate. Is that like a direct correlation? No. But is that a piece of directional information that's helpful? Yes.
[00:14:32] Noah Levin: That's like looking at before campaign and after campaign as o- as opposed to like a holdout
[00:14:37] Christina Johnson: Yeah, no, it was looking at year-over-year data actually because it was seasonal. So like with strawberries, if you were looking the month before and the month after, it might give you a, a misread because strawberries have a season. But looking at the strawberry season the year before and what the basket add was versus the strawberry season, during the campaign is what we looked at.
So first when I encountered AI, and I saw that retail media was a problem, I was like, "Huh, I bet there's a way to get all these disparate sources together with AI."
And then people started showing me their retail media results and like, for whatever reason, a lot of them are inflated. I have people who had data, and it's gotten better since, but like someone showed me something and it was like, "Yeah, I have like a 33.7 ROAS,"
[00:15:19] Noah Levin: R- ROAS is return on advertising spend?
[00:15:23] Christina Johnson: Yeah. And so there was just some murkiness in the numbers, but they were getting frustrated because you can't make a good decision on how much to spend if you can't get at least a directional read on impact. And like saying that, that their spend drove like 33x s- sales is just not within the realm of, I mean, shoot, everyone should be spending their ad dollars there.
And so then I started to play around with a smart tech dev and be like, "I bet we could get to some directionally better numbers," like based on their own sales data and like the tactics I used to use, and I bet AI could make that go faster. So that was sort of the genesis for all of this.
There's a lot of talk about, like, all these different ways to measure things, and they're valid, but I was like, I also feel like you can just get some down and dirty, like, old-school blocking and tackling that, like, AI makes faster, for you.
You can also just look at your data in specific ways. I was working with someone who ran an influencer campaign, and she was like, "Well, I can't tell if it did anything."
And I'm like, "We can also just, like, look at Google Search around the time you ran this campaign and see if it, like, searches for your item went up." It's not scientific, but it'll directionally at least say, "Hey, like, we think this drove some movement." I will die on that hill, that, you can get a good directional read from, the data that you already have.
And if you wanna get fancy, you can and should, but it's not essential, to run good marketing
[00:16:42] Noah Levin: Let's imagine for a moment, that you are sort of a, a novice performance marketer, you're running your business. Your business is not all about performance marketing, but performance marketing is one of those things you have to do, you're managing a CAC to LTV ratio, trying to get your cost of acquisition down, your lifetime value up. And you aren't yet at the point where you want to buy a product like Advisar. You know you've got Claude, you know you've got a bunch of data sources. Where would you tell that person to start?
[00:17:09] Christina Johnson: This is so embarrassingly basic. Look at your sales data. The number of people who don't actually look at their sales data along with their marketing efficiency data is insane. And where you can get sideways on that is, like, if you are running search or you're running a campaign and you're just, running for, marketing metric efficiencies, like click-through rates and, CAC like you can run a really efficient campaign that drives no sales.
And, like, the example I have for that is that someone told me... It was a big supplier, and they were running their Walmart budgets. They were trying to drive Walmart sales, and they were, optimizing a campaign for marketing met- metric efficiency, so, like low cost, etc. And they ran this, like, super efficient marketing campaign in, like, you know, LA, New York, big cities.
And then it, like they didn't drive any Walmart sales. And you know why they didn't drive any Walmart sales? 'Cause there aren't Walmarts in those cities. So it looked great from a marketing metric perspective, but the second they looked at the sales, that's a pretty big whiff. And that's where, you need to look at your actual business metrics alongside your marketing metrics, and sales, unit velocity, all that stuff
[00:18:16] Noah Levin: What's the marketing metric that looks good while sales don't move? How is that possible? Ro-
[00:18:22] Christina Johnson: Return on ad spend
[00:18:23] Noah Levin: So, what, is the R in ROAS if not sales?
[00:18:25] Christina Johnson: No, it's sales, but like it's a fraction, right? So it's like a numerator over denominator, and I can inflate that however I want. If I want to put my total sales for the time period divided by ad spend, that's gonna give me a fantastic ROAS when really the ads didn't drive that amount of sales.
So you can just manipulate it however you want it to look. Where it is helpful is if you're using the same methodology. So like if I were looking at a Walmart Connect campaign, right, and I wanted to see, which tactics were effective, I could look at the ROAS numbers in terms of, this ROAS was higher based on this methodology, so likely drove more sales.
I can't use it to say this is the exact sales number, but I can, use it in its own little set to say this was the highest performer based on this methodology
[00:19:14] Noah Levin: When you say that I have to look at my sales, do I have to look at it myself or can I have Claude look at it
[00:19:20] Christina Johnson: Yes. You can. You know, for baseline stuff, like Claude is quite good. First of all, I just am a believer, I'm a believer in using AI, but I'm also a believer in understanding your stuff before you put it in AI. So if you're just feeding your sales numbers into Claude and not looking at them, like that just opens the door for a lot of error.
The thing about Claude that I've gotten caught up on when I've been using it, just because I use it all the time just to see what it has, is it, like its training data only updates so frequently. So it's amazing at like computation. I've had conversations with like other data nerd friends and like if you know what you're doing and you want it to do the math and you tell it what to do, it's great.
For example, I was using it the other day and it, its training data ended in, I think it was like May 2026, and I didn't know. And so I was asking it something about Walmart Connect, and Walmart Connect had just changed its, measurement attribution June 1st. And so everything that Claude was telling me was wrong because its training data stops before it had that information that the attribution had, had changed.
So use with caution is what I would say. I do think you can get pretty far if it's just you and your spreadsheets. Like use it. Use it to do some analysis. It's pretty good at it
[00:20:29] Noah Levin: How do you mechanically get the data into Claude when you're using it to do analysis for you?
[00:20:34] Christina Johnson: CSV files. There's different report centers you can pull from.
[00:20:37] Noah Levin: Even in the modern era, you're still going into portals, pulling down CSVs
[00:20:43] Christina Johnson: Yeah, there are API calls for certain size suppliers, but you have to pay for them. So it just depends on who you work for. But yeah, it's still very down and dirty. Like, it's not a very souped-up system for most retail media networks.
[00:20:54] Noah Levin: Having an API would be really nice, but logging into a portal and downloading a CSV is also very easy to automate these days
[00:21:02] Christina Johnson: Yeah, Walmart will, that will shut you down if they find you doing that.
[00:21:05] Noah Levin: Will they really?
[00:21:06] Christina Johnson: Yeah
[00:21:07] Noah Levin: I wonder who's gonna win that, arms race. That's interesting
[00:21:09] Christina Johnson: Don't know. I will just tell you historically, like, because a lot of that and then scraping their site is the other thing that, but you just have to be careful. There's a lot of permissions around even using your own sales data, with Walmart, and so you just wanna be careful. 'Cause trust me, I've had that thought too, 'cause I'd be like, "I'll make a million dollars if I just have a bot that just automates, pulling CSV files."
But it can get tricky really fast, and you can end up losing access and being put on the naughty list, which you don't want.
[00:21:37] Noah Levin: So you've got a, you know, call it a folder full of CSV files. You go, you go, you know, to the farmer's market of data, and you stop by each booth, and you pull down your CSVs. Are you doing any additional sort of cleanup or merging of those data sets, before you let Claude play in that sandbox?
[00:21:53] Christina Johnson: I'm at the point now because we've been building tech that, we have purpose-built that we don't have to do any cleanup because we don't think people uploading CSVs will be wanting to do cleanup. And so there's been a fair chunk of the build that's been like take this very, very messy thing and let's put these guardrails around it and make it usable.
So that's what our back end has been trained to do.
[00:22:12] Noah Levin: What kinds of things go wrong if you don't clean up the data?
[00:22:15] Christina Johnson: A lot of the issue, like the things that we come up with are, like, if you start to compare across the data, for example, like if you want to compare Amazon data to Walmart data, like Walmart and Amazon are on different retail calendars. They attribute different ways.
And so you have to train your agents to recognize that, or else you're gonna come up with some really incorrect things. I mean, the other thing is just not every report has the same information in it, so if you're wanting your agents to look at reports, you've got to make sure you tell them where to join and that there's at least a place, like a UPC or something to join.
And so that's the kind of back-end stuff where we've been like, "Look at this to join, and then look at the data together." And it's pretty good once you do that. My poor data engineer one day was like, "What do you mean Walmart's on a different calendar?
What is this calendar based on?" And I was like, it's a 60-year-old company. Like, it's a lot of legacy things that need to be taken into account,
[00:23:07] Noah Levin: If you want to set up marketing for your company from scratch in a way where the data that comes back to you from all these different places is, understandable and attributable and chopped in the, right formats and the campaign that you're running on, channel A lines up with the campaign you're running on channel B with the same creative, are there any best practices
[00:23:25] Christina Johnson: Hire a professional. That gets really messy. Because the, thing is like, I mean, you see a lot of AI companies actually spinning up and doing this right now. It's like we can pull in all your different marketing channels and show you a dashboard with all of them and, compare some.
But it involves like understanding the nitty-gritty of each single campaign, each single tactic, so like, the bite-size example I'll give you from my world is like one of the things with comparing Amazon and Walmart, like retail media spends, is that I think it's for search.
Well, it's for all of them, Walmart has a 14-day attribution window that it'll attribute sales, whereas Amazon has a seven-day attribution window. So like all of a sudden you're not comparing apples to apples even though it's the same tactic, and you've got to figure out some way to normalize that data, to be able to compare it.
I think that there's a lot of cool AI tools out there that you can connect all your different sources to that are not gonna be that expensive, that protect you from making all sorts of human error if you don't know what you're doing. And I say that as someone who works in this space, but people are like, "Well, why don't you just build this yourself?"
Or like, "What would prevent somebody from just building what you're building with like Claude Code?" And I'm like, "Well, theoretically they could, and theoretically I could," but the realities of building in Claude or any of the other Perplexity or what have you, as a non-technical founder, A, I don't know what I'm doing.
Like it can spin something up that's pretty, right? Like it, prototyping is not a problem, but like understanding all the little details and then being able to tell the tech how to get around those details, understanding how to make sure everything is secure. I've had friends who've lost weeks trying to build something themselves, and then it keeps breaking, and then they gotta fix the broken part.
Like it, there is still a gap between your layperson and AI technology and building and a technical person that I think a lot of people don't appreciate quite yet.
[00:25:21] Noah Levin: I'm hearing you say two things that I think are both, like, so true. One is that there's this uncanny valley of software development where you can get something that kind of looks like it should be very close to a production-ready tool, but in fact, there's this, like, giant chasm of things that need to happen to make it, maintainable and secure and, you know, actually work for multiple users and, all the things that, you know, SaaS has perfected over, all these many years.
And then there's the other thing, which is, I hear you saying that there's this domain knowledge that's, like, super arcane and doesn't have any value necessarily outside of the world of retail media in your case.
But, like, the fact that, Amazon and Walmart have different attribution windows or different retail calendars, or that, you know, Prime Day was an extra day long last year, so we need to account for that.
That domain knowledge is, it might be in the training data somewhere, but it's not necessarily gonna get shaken out into the code.
If you want to build a bespoke tool set, you need to, like, BYO expertise
[00:26:27] Christina Johnson: And that's where like for me building what's been so interesting because ... So my tech team started in June, they spun out this beautiful MVP in Claude in like two days and I was like, "Oh, this is awesome."
And like these guys are, fantastic like by data engineers, like a computational biologist by training, like he knows what he's doing. It's taken a couple months to get something where we're like, "Okay, we think it's not hallucinating on the numbers now."
It's been trained with like my marketing domain knowledge. And I have like, you know, I think I even like wrote out like every single marketing question I could possibly think of, and then floated it in front of my wife who's been a marketer for 20 year- Like so it's, it's ... You have to really program it with the human knowledge, which I think a lot of people, including myself, sort of miss.
I've found AI tools are very much a reflection of the humans who've created them, for better or worse. And like I see it in my own tool where I'm like, "Oh, that is my fault that I just like created in digital form for somebody else," which has been really interesting.
[00:27:26] Noah Levin: Tell me about the experience of recognizing that there's a problem that could be solved with AI and then actually building an AI product to solve it
[00:27:36] Christina Johnson: What I'm finding, at least with what AI is at the moment, is it's like an incredibly powerful tool. I'm not taking anything away from that. But, the detail work that needs to go into creating something that's actually useful, and then the customer work in terms of understanding what customers want, is actually a pretty long and, like, grueling road to get to something that is fully useful.
I never developed true software, I can only speak from, conjecture, but I would imagine, like, the discovery p- process you went through with that kind of software still exists today. You can probably iterate faster after it, but people are still at the core of everything, and so understanding what people want takes a long time, and then making sure you're not giving them bad information also.
Like, gives me agita every night that I'm gonna give someone bad data. And so that's been a real focus of ours, has been, like, data governance and guardrails. And so that has taken a while. Worth it, but it's not the, like, snap your fingers that I think everyone was believing, you know, a year ago.
[00:28:37] Noah Levin: What were the major proof points along the way? I, I imagine you had to do some of this work solo before going and raising money, going and finding the technical counterparts. Like, what were you able to do on your own to kind of build the proof of concept and, and validate the idea before you kind of took the leap?
[00:28:55] Christina Johnson: So I was really lucky, and I actually was in an accelerator that had free tech help, and so I had a tech dev who was pretty AI savvy early on. But the second we were able to get some data and we, like, built something around it and I started showing it to people and they were like, "Oh my gosh."
And it wasn't even that sophisticated, but it was just being able to look at massive amounts of data and have it be distilled for you in a usable way, like, got an immediate reaction from people around here. So that was like the first proof point where I was like, "Uh-huh, this is like, this is like a thing."
And then trying to figure out, like, where I personally had something unique to add to it. We haven't touched on it much here, but like I'm very aware of like the differing incentives of the different parties in retail media and getting people to spend money. And so you can see that play out in numbers.
The whole reason I created Advisar is 'cause I was like, I feel like suppliers just need a, like, third party that can tell them these numbers are correct or these aren't, actually action on these numbers, and here's how to make a good confident spend in retail media and/or adjust your campaign better without garbage of like, "I'm trying to get you to spend more money. I'm trying to get you to..." Like, 'cause both agencies and Connect are, a lot of them are trying to do that because agencies are charged by amount of spend you spend with them. They charge a percentage, and Connect is just wanting suppliers to spend more money. And so when I started articulating that thesis and it resonated with people as well, and I saw that the tech could actually do it, that was my other thing is like, I can say this.
Is the tech actually gonna be able to like, get to a more rationalized number? That was another proof point where when I started showing it... Like, I just, have been showing suppliers. I mean, I live in Northwest Arkansas, so like, two doors up the street is, is a supplier. So I just show as many people as I can to get as much feedback as I can, and that's been the thing that's validating all along because nothing else has really been easy.
Like, raising money is challenging, building an AI is challenging, but, hearing people continue to articulate the same problem was the thing that kind of kept me, kept me going.
[00:30:50] Noah Levin: What's the biggest thing about the product and the shape that it's taking now that, would surprise you, six months ago?
[00:30:56] Christina Johnson: I don't want to say it's so easy to make bespoke, but like we can program business context in cross data, and then we just built a function where you can program your own questions that you want to have the agents react to every week. And it's like how bespoke these things can be made for like not that much money is exciting to me.
People are like, "Oh, what about tokens? What about how long it takes to develop?" And it's like, it's really ... Like building AI native is really nimble in a way that like I don't think I've ever seen, and that I wish I as a marketer had the ability to build something that bespoke and then be able to ask it questions and get like real answers to my own problems instead of having, like I said, trying to like run around the marketing floor at Walmart, trying to beg people for data.
I also am having the experience of, like, I can build anything. Isn't that a superpower? I feel like a wizard when I sit down and try and build software for myself. I also think there's, a downside to losing the constraint of work is hard, resources are scarce, need to be prioritized, which is that it's so easy to build everything that you don't have to do the upfront, editing to know that you're building the most important thing, and software is less opinionated as a result.
There's a right way to use, software that only has a few features. But if every B2B customer gets to ask for its own version of the same thing, you end up with this sort of feature sprawl
[00:32:22] Noah Levin: And you see this in some of the AI tools, by the way. We talked about this on an earlier episode of the pod, but there's, there's Claude Cowork and Claude Code that do very overlapping things. And it's kind of because it's possible for these platforms to build so fast that they were able to build what turned out to be two very much the same things at the same time for different audiences.
The product manager of these tools was asked, like, "Is there a right way to use these? How do you think about thing A and thing B?" And she was very transparent. She said, "I think that's the consequence of moving fast, is we kind of built things without a coherent vision." So do you find yourself, doing much upfront editing?
[00:33:00] Christina Johnson: So for me, like I had to do a lot of editing upfront because first of all, my fundraising took like way longer than expected, and I'm not independently wealthy.
And so I wasn't resource constrained in terms of the tech, but I was very resource constrained in terms of money. And so that made me get very focused, because I was paying people by the hour. So I was like, well, I needed to do these things.
[00:33:23] Noah Levin: Why was fundraising hard for you?
[00:33:25] Christina Johnson: First time founder, pre-seed, which means I ha- I didn't have clients at the time. I do now. Live in Arkansas. I'm a woman. It's a tech field. I mean, like, and fundraising is just hard. The investors I have now are great and the right investors, and it was, it was a long and painful journey, but, like, it wasn't a slam dunk.
In this whole fundraising time, like, I started out with, someone who agreed to be, like, a founding CTO, but, like, didn't want to stay on very long. So he built me my initial MVP, and I think what I hadn't realized was how good he was with data. And so he created this great, very credible MVP.
He left, and I ended up using, a tech dev who was also very good, but, like, zero data background. And so she ended up, creating a next gen version of one of the MVPs, and I was just like, "Yeah, these numbers look good." Like, like anybody, right? "These numbers look good. I think it's great.
Let's show it to the client." Ugh. And he looks at it, and he's like, "This is not correct." And then, like, starts asking, like, the chat function questions. And he's like, "And this is not correct." And he's like, "I can't even give feedback on this because the numbers are so wrong." And it just goes to show you, like, that's the other caveat for people like me who are operators who are not technical people, is, you don't know what you don't know.
And so, like, there's so many different skill sets in the tech world as well. Like, and both my tech devs were good, but in very different ways, and, like, I hadn't realized it, and then I hadn't done the checks on the back end.
[00:33:42] Noah Levin: When you look ahead and you think about what this product that you're building is gonna become and what's possible with the technology you're using to build the product, what do you think is gonna be different about being a marketer in a year or three years?
[00:35:14] Christina Johnson: There's a lot of talk, and I think you've mentioned too, you've seen this in your project work, like a lot of people are going to, agentic buying, for digital media. I don't know what I think about that yet. I think it definitely has a place, and I think there are ways that, that it can help.
But I think that, again, it's the garbage in, garbage out if you don't program it to take certain nuances into account. If you don't take it, like it can, it can make mistakes. So I think there are gonna be things about marketing that are faster, like that, like being able to analyze data, but I do think the core skills are gonna be there.
Like, I still think you need to understand who your customer is. And that is you. Like, the AI is not gonna understand who your customer is. I think you're still gonna need to ask the right questions of like, what is this campaign meant to do? Why am I doing it this way? And I think when you start to automate everything, you lose some of the nuance of like the human understanding aspect of it.
Like, I've talked a lot about how, like I reflect a lot on how like AI reflects my own flaws back to me, like strengths and flaws. I think that's gonna be the same thing with marketing. You're still gonna need to have those marketing skills. You'll just have more data to act on them with. You'll just be able to do things faster and react faster. But like that core competency of understanding what the marketing is meant to do and how to reach people, I think stays with humans.
[00:36:29] Noah Levin: I think I agree with you and disagree. I'll try an angle on for you.
Okay, here's where I disagree with you. I think that, life is just a series of prompts, the AI is always gonna be the middle mile with context sitting on one side and some sort of an output that you do something with, the action, right, on the other side.
AI is the computer in the middle. It's doing some computation on, on whatever you tell it. And so whatever is the thing that the AI, can't understand, maybe it's, who is your customer or what is strategy for reaching that customer, maybe it just needs, the context a step earlier that you're using to make that judgment, and it can do that part too. And whatever the action is, like go make the ad buy,
[00:37:11] Christina Johnson: Yep
[00:37:11] Noah Levin: It just needs to understand that context and it can go, it can go a little bit farther down the chain, and you can kind of grow it from whatever it's doing now outward in both directions. So I, I think that, if you look really closely at whatever it is that you're doing before and after the AI gets involved, it kinda looks like context in, as a prompt, action out, even if you're the one who's doing the computation. But then I'll steelman your point of view, which is, if everything reduces just to, AI doing the work, there is something missing at the end of that process. There is a role for the human element in, taking points of view and taking accountability and being credible.
And like this podcast as an example, there's a reason that you and I as humans are showing up on this podcast to present ourselves as the avatars of the work we're doing. And so there is a role in at least some businesses for the person to come in and inject their subjective point of view and kind of like, you know, kick it off-kilter a little bit.
Mechanically, I think the AI can probably do all of it
[00:38:11] Christina Johnson: So I guess where I get stuck is AI doesn't learn from experience, in the same way that we do. So like the AI is like not in the store observing how people are shopping, there's a piece of like the human experience that it just can't see, at least now. And like, you know, I fully acknowledge that I don't know what's going on in Stanford in some AI lab and, and who knows, right?
But I guess what it reminds me of in a certain way is when everyone was like, online learning is gonna take the place of schools. It's just so much easier, so much faster. And like what happened is it really became a hybrid and certain people were missing that full human experience 'cause there's like an intangible there that the online doesn't capture.
And like this isn't a fully formed thought for me yet, but there is an intangible of the human experience that like the AI can't quite capture that you and I experience being out in the world that we can then feed into the AI. But I don't know that I'm bought into the AI's gonna be able to replicate that without us feeding it.
[00:39:07] Noah Levin: All right. Well, we'll do a follow-up episode in 10 years,
[00:39:10] Christina Johnson: In 10 years when the, it'll just be our avatars
[00:39:12] Noah Levin: That's right. Uh, I, I'm s- I'm, I'm just saying it out loud now, and I'm sure my agent will call you in nine and a half years and schedule it. So bringing it back down to Earth, just to close us out, if you're a marketer listening to this podcast and you, want to do one concrete thing to use AI to make better decisions around your spend tomorrow, what is that thing?
[00:39:36] Christina Johnson: Learn how to get all your data. Like pick a campaign, pull your sales data, pull your, marketing, data. Learn how to develop a really good prompt to ask what happened in the campaign, and just use Claude to analyze it.
Like, it doesn't have to get super fancy. That's my other thing is like AI can do a lot in just its basic form. So yes, I'm building an AI capability for like more advanced stuff, but you can get really far with like Claude and your CSV files, and you shouldn't be afraid to use it. That's my like very basic advice
[00:40:05] Noah Levin: Good advice. Who should be reaching out to you about Advisar?
[00:40:08] Christina Johnson: Brands or suppliers who are selling into retailers and they're spending in retail media and they want another opinion, they want help organizing their data, they wanna pick my brain. We're in beta now. I'm looking for another beta partner.
But really, like any questions around retail media, we're a good source. And if I don't know the answer, I'll send you to someone else who does. That's my other... I'm not gonna claim to do stuff I can't. So if you, if you really wanna have your incremental holdout test, I will send you to s- people who do that.
So
[00:40:36] Noah Levin: The moment I have any reason to calculate ROAS for my life, I will be calling you immediately.
Christina Johnson, thank you for coming on "Serious People."
[00:40:44] Christina Johnson: Thanks, Noah. Appreciate it.
[00:40:46] Noah Levin: Thank you for joining us for the Serious People podcast. If you like what you heard, subscribe on YouTube, leave us a review on iTunes, or sign up for our newsletter at seriouspeople.ai/podcast. When we're not podcasting, Serious People helps businesses put AI to work in their daily operations. Visit us at seriouspeople.ai to learn more.
The Serious People podcast is sponsored by Valet.dev. Go try building your own agent today at valet.dev or click the link in the show notes. See you next week.