[00:00] You can only really leverage MTAfor making decisions if you are doing some type of rule based attribution. maybe you have a, how did you hear about us? [00:00:11] survey And then you use that data as an alternative data source to also include that in your MTA. Or I've also heard of, trying to, estimate clicks based on other data and then put that in the MTA model. But just looking at the clicks,it's going to give you this false confidence. [00:00:31] that you have an overview of the customer journey [00:01:05] About Barbara --- [00:01:05] Phil: Barba was an early employee at Her, a YC backed startup and became the biggest platform for LGBTQ women where she would eventually become head of growth. She was also head of growth at different startups like Parity and Home Run, and she worked at an agency where she led data and analytics for Microsoft EMEA. [00:01:23] Then she went out on her own as a GTM and analytics consultant for various companies like Gitpod, WeTransfer, Sidekick, and many more. dbt labs. She also has a newsletter on marketing data called zero 21 newsletter. She produces content for data brands like dbt, mixed panel and amplitude. And, uh, she has a lot of case studies, webinars that she does for them. [00:01:44] So super excited to chat with you today, Barbara. Thank you so much for your time. [00:01:48] Barbara: Thank you so much for having me. Very excited about this. [00:01:51] ​ [00:01:51] Phil: [00:02:00] [00:03:00] I've been down the multi Dutch attribution versus incrementality conversation with a lot of guests on the show in the last like two months or whatever. So I hope the regular listeners aren't annoyed by, by this topic, but it's, it's definitely been this like. For me, uh, my last company had a ton of internal debates about this. [00:03:59] And I [00:04:00] think we definitely could have leveraged an external consultant's perspective on this, because a lot of the conversations I've had have been from the vendor side, and I'm really curious to get your kind of like, Unbiased unattached to a solution take on, on some of these things. [00:04:14] Building Data Literacy Through SQL --- [00:04:14] Phil: But maybe we, before we, we get super deep into that, um, I love asking this kind of like data literacy, data skills question, you're really well positioned to give advice on how does someone get to Barbara's stage in like three, four, five years, like, let's say someone's a couple of years into their marketing career and they come to Barbara and they're just like, I don't know. [00:04:36] Don't think that I'm like quite there on the data, the analytics side. Like, what do I need to do today to like, get much better when it comes to like a data literacy for like leading my team or just like just becoming better at marketing, what's, what's your advice there? [00:04:53] Barbara: Yeah, that's a great question. I think for anyone that really wants to understand marketing better or [00:05:00] leverage data better for marketing, I think SQL is a great place to start. I think even, you know, if Language learning models, even with all these tools that, you know, have these nice UIs where you can drag and drop. [00:05:16] I think knowing SQL really helps you understand how data is structured and how you can join different datasets, how you can group. So I think if you're trying to get on that data ladder and get better at it, you know, just try to start using SQL for some small use cases. I would say you probably don't need to go out and like do a course, but almost every company nowadays, you know, has some marketing data in a structured warehouse that you can query, you know, maybe it's your amplitude events that are being stored on a, on a data warehouse. [00:05:55] Maybe it's Google Analytics that's on BigQuery. You can start to [00:06:00] create some reports and then get better with time and Apart from that, and I think we'll cover this a lot as we talk about attribution, but understanding how the numbers you're measuring are measured is super important. You know, um, uh, when you're looking at, you know, reporting on, on Meta, reporting on Google Ads, reporting on Google Analytics, understanding why certain campaigns are getting the credits. [00:06:29] And why certain campaigns are not is, if you understand that, I would say you're already in the top 10 percentile in terms of data driven marketers. [00:06:40] Phil: Yeah. It's one of my favorite newsletters where you unpack why certain numbers are different across platforms because they are measured differently. And so the expectation is that it is different. And so, yeah, when you're looking at a report in your marketing automation platform and a report in GA and a report in mixed [00:07:00] panel, like all of these tools are using different. [00:07:03] methodologies to measure and the expectation is that they won't match one to one. You need to decide when you say web visits, or when you say conversions, you need to pick and make it obvious what that definition is and make sure that the tool you're using to measure that number matches your definition that you're kind of picking. [00:07:21] But yeah, it's a, it's really good advice. I love the. Um, the, the advice about, um, understanding more about how data is structured and where you're joining stuff. I think a lot of companies are becoming more warehouse centric and, and, and marketers maybe don't have full access to, to data warehouses, but, um, you know, census sponsor, they show they have access to like, uh, starting to build, uh, segments and understand more what data they have access to that they maybe don't have in, in some of these tools. [00:07:52] So, yeah, I think it's, uh, it's great advice. Those skills aren't going to, uh, die anytime soon, even though LLMs like Chachapiti [00:08:00] definitely make it easier for folks to like, understand stuff. Like I like taking a piece of code and pasting it and Chachapiti and like, what's going on here exactly. Like, why doesn't this query work? [00:08:12] And, and so that's, that's a great way to learn too, but yeah, I really appreciate the tips there. [00:08:16] Rethinking Attribution Beyond Click-Based Models --- [00:08:16] Phil: Um, let's yeah, let's, let's dive into the attribution. So just for the folks, uh, we're. We're going to be doing kind of a roundup episode featuring different takes and perspectives on multi touch attribution and incrementality and, and MMM, and I've had a lot of folks on the vendor side, give us their take about, um, some of these topics. [00:08:35] So I'm curious to read out like some of the summary of answers and just get you to give your perspective on some of that stuff, and maybe we can start with attribution MTA specifically, lots of folks. On the show so far have claimed that attribution is misunderstood as a direct cause and effect in marketing. [00:08:54] For example, if someone clicks on a Google ad and then converts a lot of MTA tools, especially [00:09:00] tools that leverage last touch will give a lot of credit to SEM or that Google ad, uh, but they don't necessarily reveal what prompted the search in the first place. Maybe I heard about it on a podcast or maybe it was a TV show. [00:09:13] Dark social, like a Slack community recommendation. We don't really know what caused that person to Google something and then click on the Google ad. Do you agree that MTA isn't made for cause and effect and is more of like a behavioral analytics, which tracks actions without proving one cause the other. [00:09:30] What's your take there? [00:09:32] Barbara: I feel like I would even take a little step back, and I would start with, how are we defining attribution? Because it feels like what you're saying, and what a lot of people are saying, is that is when they're thinking about attribution, they're thinking about click based, touch based models. And to me, attribution is a lot bigger than that. [00:09:54] And MTA and click based, you know, touch models is one way that you can attribute. [00:10:00] To me, attribution is about understanding the results of campaigns. And that's how I would put it. And there's, you know, multiple ways that you can understand those results. And we will cover, you know, like, holdout tasks, incrementality, MMM, all of that. [00:10:14] And click based, touch based, MTAs. Is is a little, is a little part of that. It's not the, the whole picture. And I, I, I, I would say that, yeah, I agree. I don't think it's necessarily about, like, call MTA, it's about cause and effect. I, I'm, I'm currently, you know, I'm working on an attribution course that is actually going to, well, I think it will be happening by the time the podcast [00:10:44] Phil: Okay, cool. We'll make sure to link that out. [00:10:46] Barbara: So the first cohort is already full, but you know, people can then, you know, join the second cohort. But I'm organizing, um, the attribution course with, uh, Tim, with, uh, data engineer called Tim De Chow. And one of the [00:11:00] conversations we've been having a lot is what is the role of MTA in 2024, in 2025? Like, what can it actually help you uncover? [00:11:11] And the reality is, you know, like MTA. just with clicks, just with tags, like the MTA you see on, on Google analytics or you see on your event analytics tool is really only good for measuring a certain type of campaign. So that's why you were saying, you know, attribution tends to maybe over credit search marketing. [00:11:37] That's because MTA is, at the end, really good at measuring search marketing. And that's pretty much it, by itself. Because MTA only works if there is a click, and within, you know, a short window after that click. there is a conversion. So when you're thinking about, okay, what kind of [00:12:00] campaigns are, are, are at the last stage of the funnel. [00:12:04] So we're bottom funnel campaigns with people with high intent and need a click in order to work. And when you, when you put it like that, you see like, okay, it's. It's search. Because, you know, if you do like, uh, video, that doesn't generate clicks. If you do display, again, most people don't click, they've done analysis and there's almost no correlation between CTR and sales for social and for display. [00:12:31] So when you're looking at click base, you're pretty looking at a good way to measure search. And not anything else. [00:12:40] Phil: Gotcha. That makes a lot of sense. I like how you've coined taking a step back here and like attribution is not equal to multi touch attribution. Multi touch attribution, AKA click based attribution is only one little thing in a bunch of different methodologies that fall under the attribution [00:13:00] umbrella. [00:13:00] What do you think is the difference between. This, this umbrella of like attribution and measurement, like, are they the same thing? Like if we say incrementality and MTA are methodologies to measure the effectiveness of your campaigns, is that like the same thing as saying like their attribution methodologies? [00:13:18] Like what's the difference for you? [00:13:20] Barbara: Yeah, I would say that's the same. I guess when I think of the word measurement, I think of perhaps, you know, other insights that are not necessarily just related to understanding the results of campaigns. So I would also put, you know, maybe I want to understand which messaging is performing the best with my audience. [00:13:39] And then I would put that under the measurement umbrella, not under the attribution umbrella, but the attribution umbrella. I would say attribution is the, the act, the science of measuring, you know, business results from your marketing campaigns. And that can be [00:14:00] leads, revenue, sales, it can be whatever metric you're using to move the business. [00:14:06] Phil: Gotcha. Super helpful. [00:14:08] Limitations of Multi-Touch Attribution in Credit Distribution --- [00:14:08] Phil: Um, let's talk about one of the ways that click based multi touch attribution is useful. Like one of the. Ways MTA has been described, uh, by another guest is a tool for distributing credit across different touch points, not determining causality. It helps understand customer engagement and you can assess whether channels like OTT or CTV are contributing to conversions. [00:14:35] But not necessarily proving direct cause. What do you think about empty as a credit distribution mechanism? Do you agree? [00:14:44] Barbara: I don't think it's a very effective credit distribution because like I said, it can only credit, it can only include a touch point when there is a click to it. So in the end, the, the [00:15:00] view that you're going to have is going to be a view for only the strategies that generate clicks. So it's not really going to tell you, you know, what was the role of, yeah, even like CTV, the role of social, what was the role. [00:15:13] of, of, of these other touch points before. Something that, yeah, we've been talking about also, you know, like in this, in this course is that I think nowadays you can only really leverage MTA for making decisions if you are doing some type of rule based attribution. So let's say in that case, maybe you have a, a, how did you hear about us? [00:15:42] survey that you ask customers when they make a purchase. And then you use that data as an alternative data source to also include that in your MTA. Or I've also heard of, you know, of data agencies that have been trying to, [00:16:00] um, estimate clicks based on other data and then put that in, in the MTA model. But just looking at the clicks, it's, it's, it's going to give you this false confidence. [00:16:13] that you have an overview of the customer journey when you don't. [00:16:19] Navigating Attribution in a Multi-Channel World --- [00:16:19] Phil: It is, do you think there's a way to make up for that somehow? Like record box has built this, like they don't claim to, to solve like the third party thing. Like it's well documented that empty is reliance on third party cookies is flawed for sure. And they don't apply to offline channels like TV or direct mail, but some vendors Like record box claim that modern MTA uses first party data, probabilistic methods and partnerships with platforms to access deterministic data. [00:16:50] So for these offline channels that don't have clicks, like you just mentioned, um, like we have models, we have surveys to connect that data, to measure performance still in some way. And [00:17:00] the focus is on using granular data or modeling, depending on the channel. Like, what do you think about that? [00:17:07] Barbara: I think it's possible. I don't know necessarily what's a good way to validate that data and how to keep on improving, but I think, you know, there is a lot of different data sets that you can use to estimate. if there was a casualty to the action. So, you know, like for example, like, you know, MMM will look at, um, uh, you know, the impressions they have generated from a channel on a daily level, and then it will look at the, the sales results on a daily level. [00:17:39] And then based on that, it will use a casual interference, you know, like a Bayesian model to, to understand if, there was an impact. So you would be able to take that methodology, but then instead of trying to correlate to direct sales, you can try to correlate it to clicks and [00:18:00] then have like a syntactic click that shows in your MTA model. [00:18:05] But I think, I think this is where we're moving towards, [00:18:10] Phil: hmm. [00:18:11] Barbara: but I think that we're still really trying to figure out what's the best way. With the, with the course, you know, we've been having a lot of conversations about how these things have changed. And I think, you know, like back then, you know, even like before, way before, like these privacy changes and the appreciation of cookies, I think MTAs worked better. [00:18:37] Because you had companies that were, you know, mainly relying on one form of advertising. You had companies that were running, you know, mostly on page search. And if that's the case, then indeed, you know, like MTA works. But nowadays, At least, you know, every client that I work with is, is running, you know, like, yeah, a combination of channels and strategies. [00:18:59] And [00:19:00] MTA was never really a fit of that. Even if you still had, you know, like, third party cookies, even if you still had, you know, no GDPR, all of that, you know, MTA would still, you know, Have all these gaps that you cannot really use to understand the user journey across all of these different types of marketing strategies. [00:19:21] I, what I always try to do when I'm talking about attribution, I always like to look at attribution from a strategy perspective. So I'm not, you know, most of, I'm not going to a client and I'm going like, this is how you need to attribute. I'm looking at, you know, okay, you're doing meta. How do we attribute meta? [00:19:42] You're doing alongside meta, you're doing podcast sponsors. How do we attribute that? Really try to separate because each type of attribution is the best fit for a certain type of marketing strategy. So that's why you can't really answer these, these, you [00:20:00] know, holistic macro questions with just one data source. [00:20:05] Phil: Very cool. I like the, the strategic channel based approach to, to attribution there. [00:20:10] So what, like, what is MTA really good at aside from like click centric channel based things like, like meta and Google, like, [00:20:21] The Limitations of Multi-Touch Attribution for Conversion Path Insights --- [00:20:21] Phil: do you think that there is something to this idea that MTA is really good at the path to conversion, like showing you the paths of new versus returning customers? [00:20:31] Obviously. Only limiting that to, to, to online touches, but is it good at like how channels are impact conversion speed, or like you kind of answered this one already, like overall marketing effectiveness, like does this idea of path to conversion kind of jam with you a little bit with MTA or no? Yeah, [00:20:51] Barbara: MTA is not particularly good at. Because, you know, when you're looking at social, especially nowadays, most clicks happen on [00:21:00] mobile. So even something, even if you're, I had a client just now, you know, they're a DTC brand, they get a lot of mobile conversions, and then they want to use like, you know, UTMs to measure, you know, meta ads. [00:21:13] But what I told them is like, it doesn't really work. Because when someone clicks on an ad, On the Facebook app, you know, Google Analytics or whatever tool you're using to to use this is going to understand that as a different user as the user that goes on to open Safari on their phones and makes the purchase. [00:21:34] So unless they're literally clicking on the on the Facebook ads on the Instagram ads and making the purchase like within that session without ever leaving their Facebook and Instagram app. A click based UTM model is not going to capture it. [00:21:52] Phil: and that's not even capturing people that are using cross devices like will go on on [00:21:58] Barbara: the same device, [00:22:00] literally the same device, but it still can't capture it. So the gaps, the gaps are just so wide that you're going to be looking at this data and you're going to go like, yeah, but is that really the case? No, like I've had also like a client where we invested, we used Snowplow, and then we invested a lot in MTA. [00:22:18] And then we were, you know, what it told us, and we could have like different attribution models was that most people that were searching for the brand name on, on the page search were converting within the first session. Like that was their first visit. Obviously, that's not the case. Because, you know, like if you're searching for the brand name, like I'm going to guess. [00:22:43] that you've gotten some other advertising from us before and you're also converting and purchasing within the first visit like that's all can be true right so you have to take all of this data [00:23:00] yeah with a grain of salt and it's our job as marketers to go like yeah you know that doesn't seem like it's really it. [00:23:06] Phil: Yeah. Yeah. Yeah. And so maybe we can talk about like what what is what are some of the alternatives that, um, You kind of work with clients on not that like MTA in and of itself or the click based stuff, like there maybe is some value, but only when you're combining it with some of these other methodologies. [00:23:26] Is Incrementality the Golden Alternative to MTA? --- [00:23:26] Phil: So one of the ones that's been painted as the golden alternative to MTA is incrementality. And so it's been defined as measuring sales that wouldn't have happened without. Marketing efforts. It's like getting a baseline to avoid attributing everything to a certain channel or like marketing. Right. And the focus should be on understanding the marginal return of the last dollar spent. [00:23:50] Um, and like, while methodology is like MTN, we'll talk about. MMM in a second, like they all have pros and cons. The key is to start with the right question, not the [00:24:00] methodology itself. So incrementality has a bunch of testing involved, right? And we can unpack some of this stuff, like geo based testing, holdouts. [00:24:08] They can all compliment MTA reporting, helping marketers optimize spend, um, based on like real data rather than relying on models or assumptions. What do you think about incrementality? Like, is it. Being put too much on a pedestal. Like, do you agree that it's really the, the golden alternative to MTA? [00:24:28] Barbara: you know, what we're going to unpack in this conversation is that there is no golden alternative. There are, there's a lot of solutions. Some solutions work well for certain strategies and all of solutions have different gaps. I, I actually really like incrementality. I always try to tell clients to, you know, have a testing roadmap. [00:24:54] Where you are always running incremental tasks. But yeah, but that's, that's very difficult, [00:25:00] isn't it? You know, um, a lot of the times, you know, like, yeah, marketing teams have targets, you know, leadership wants campaigns to run. Incrementality sometimes means like, yeah, actually get having a cut of your revenue. [00:25:13] If you stop something that was working, not all strategies can be run incrementality, can be run incremental, you know, like, If I'm trying to assess, you know, what brings me leads, I can't make this podcast an incremental strategy, you know, say it, Hey, Phil, please only, only put this, you know, in these, on these, um, on these locations so I can analyze the impact, you know, some, some, some marketing campaigns are very long term. [00:25:43] So you're not going to get the insights there. Some marketing campaigns have small returns. So even picking a geo can be difficult. There's all of these scenarios where incrementality doesn't work amazingly. But I [00:26:00] think, you know, if you're, if you're running social, if you're running display, and if you're running that, you know, in, let's say, like a six figure budget on a monthly basis, then I think you should definitely try to have incremental tests in your testing roadmap. [00:26:16] But yeah, that requires a dedication from the marketing team and support from the data team to help them analyze, find the MTAs that they should be running in, all of these different things that sometimes teams don't have the resources to do. [00:26:32] Phil: Yeah, definitely. Even startup teams. [00:26:34] Balancing Experimentation, Measurement and Execution --- [00:26:34] Phil: Like I, um, had a conversation earlier in the year with, uh, one of the guys who leads growth at notion and he was working on Spotify, Spotify before that. And they had, um, All of these like growth pods within a massive like enterprise at Spotify. Right. But there was like all these different like cross, um, departmental pods essentially, and like a designer with a [00:27:00] marketer and a data engineer, and then a data scientist, they're all coming up with like experiments to drive a certain KPI or a certain area of the funnel. [00:27:08] But in like your average company, your average startup, like. Marketers just have a ton on their plate and coming up with a testing and experimentation roadmap, as amazing as it sounds like there's, there's a lot of other stuff that takes priority over testing every single thing to get a like this incremental causal confidence that this thing that we're doing is leading to revenue. [00:27:31] So yeah, it's a, it's a tricky thing for sure. [00:27:34] Barbara: It's, it's always a balance, right? Are you, are you better? I mean, you should always try to invest to understand what has worked, but there's also also situations where you're better off. Just keep doing it. Than trying to measure and all of, yeah. All of these things, you need to put it on a balance. [00:27:56] Understand what's the impact, what's the cost, and [00:28:00] try to do the best you can. [00:28:02] Phil: Yeah, [00:28:03] ​ [00:28:03] Phil: [00:29:00] I've actually chatted with folks at the companies like Ahrefs and Wistia that aren't like fully allergic to measurement and attribution, but like they really don't focus on it. It's not anywhere near the top of their list of priorities. Um, Ahrefs doesn't even use Google Analytics to measure anything. [00:29:55] Like the only real numbers they look at is [00:30:00] And if revenue is going up and it's green, they're doing something well. Like it was with a Sammo who's leading content there. So he's really big on YouTube and he was looking at a lot of like sentiments, analysis, and like comments in YouTube and really optimizing like the videos. [00:30:17] But he was like, am I measuring how many views are leading to conversions? And none of that stuff, like he's looking at revenue and like Tim Sulu, their, their CMO is just like really big on, on that idea also. But I think that's like putting it. A step like more extreme than what you're kind of advocating. [00:30:36] Like there should still be some measurement, especially if you're like trying new ideas and you don't really have a good grasp of like what is leading to revenue and maybe you're not as grown as fast as, as HRS and, and, and Wistia, but there's still should be some measurements in your opinion. Right. [00:30:55] Barbara: And you need to be able to grow with confidence, right? There reaches a, there reaches a point [00:31:00] in a company's life, be it because, no, they've gone public, be it because of, you know, pressure of the board, that they're going to want to know, like, what does the marketing budget need to be for us to grow 15 percent next quarter? [00:31:13] And if you don't know where your results are coming from, you're not going to be able to answer that question. [00:31:19] Phil: Yeah. Especially when we're like, we want to like two X or five X revenue. And then they're asking the CMO or the marketing leader. What are like the three things that led to the most revenue? Can we just like hire 10 more people and just like do a ton more of that stuff? The thing that's different with like hreths of the world is that they're not trying to like 10x the size of the company to become like a billion dollar unicorn. [00:31:47] Like they're happy. With the size that they're at right now with the profitability that they're at. So like, not every company is trying to 10 X and has like this big board of VCs that are like pushing for that, like crazy growth. So [00:32:00] yeah, it's, it's a different balance everywhere, [00:32:02] Comparing MTA, MMM, and Incrementality --- [00:32:02] Phil: but I wanted to ask you, so. [00:32:05] MMM, incrementality, MTA, like we talked about, there's, there's different areas where each of those methodologies shine. And maybe we can chat about like what all three methods are useful for, but where they focus, uh, where they differ in their focus areas. So. This is kind of a summary of, uh, the, the takes from different folks on the show. [00:32:26] And I'm curious if you agree so we can unpack them one by one, but I'll, I'll just like, um, I'll read it out for you for, for all three MTA offers path to conversion insights, better seen as behavioral analytics that optimize. Bottom of the funnel conversion journeys. MMM requires at least two years of data and 5 million annual spend. [00:32:48] It helps with high level budgeting, but not daily optimization. It offers correlation insights showing which channels are linked to business outcomes, but doesn't prove causality. And. Incrementality [00:33:00] testing, while it has some of its downfalls and potential costs related to it by turning stuff off, it helps determine causality. [00:33:09] It shows the last dollar spent is impactful, but you know, point in time results do have their challenges there. Do you agree with those three? Like, is there one that you want to double down on? Like curious, your take. [00:33:22] Barbara: I think roughly I, I all agree with them. I don't know exactly, um, how much budget is required for MMM to, to make sense, but I agree there is a significant budget threshold because it is, you know, a casual interference analysis in the end. So we need to have, you know, enough data to be able to, to make this correlation analysis. [00:33:51] And I think like I've said on MTA, yes, bottom funnel, but also important to mention that there needs to be clicks there because [00:34:00] perhaps you're doing some bottom funnel activities that don't have clicks like I don't know, case studies, for example. It's a bottom funnel activity. And indeed, you know, it doesn't mean that they're not important, but they're not going to show it like in an MTA model. [00:34:16] And the other one was the one incrementality. Yes. Yeah. And I would also add, there's also the, the The length that it's important there too, you know, how, uh, how long does it take for the conversion to occur? But that's usually for when people are doing like an incrementality analysis, they will also want to pick an event that's a bit more higher occurrence. [00:34:42] So maybe you're not looking at sales, maybe you're looking at leads or white paper downloads. A lot of these things B2C because they're just so much more Data, [00:34:56] Why Startups Should be Focused on Pouring Gas in the Fire --- [00:34:56] Phil: Um, for startups in, in measurements, like [00:35:00] obviously many startups end up failing because they're testing way too many channels and they're trying to do too much stuff early on. And, uh, they're expecting a measurement company to fix like lack of results, uh, based on like when they're, they're, they're bringing this person or this, this vendor in. [00:35:17] But measurement is like a scale. It only focus and reflects on like, what's working in a sense. Like once you have a profitable scalable channel, then you can consider adding measurement to enhance growth. Do you agree that like, that's the typical approach or recommendation for startups that like they really shouldn't focus on attribution, let alone engage with measurement companies when they're too early on in that journey and instead they should rely. [00:35:45] Okay. On intuition to grow revenue and focus on a couple of key channels. And once they're scaling, then they can use geo based experimentation, marketing efforts, or like, do you think that they should focus on finding just one [00:36:00] channel? Just like curious, your take on like, what should startups be doing that? [00:36:03] Like they don't have a data scientist team. Maybe they don't have the resources to hire a vendor or like a measurement expert, like you would, what advice would you have for startups? [00:36:12] Barbara: I think there is a lot of different questions there There is you know a question on marketing strategies and what should they focus on there is a question on measurement So let me try to like unpack I guess the first thing that I would say, you know If you're a startup and you found a channel that is working for you double down on that You know, I'm a firm believer that the best marketing strategy is to pour gasoline into the fire. [00:36:41] Phil: Very cool. [00:36:42] Barbara: You know, whenever like a company, you know, they're like, Oh, we want to do pay media. I try to understand how they're growing today and how paid media can help them increase that. Not about starting a whole channel, but how can we use paid to improve what's already working? [00:37:00] So I would say first things first, you know, you found your channel, squeeze all the juice. [00:37:06] Because, you know, channels change. You know, competitors enter the market. There is, you know, privacy changes. So it's not, you shouldn't take it for granted that that channel is always going to be there for you to grow from it. So make sure to, you know, yeah, squeeze it all. What I would also say is, I mean, and I would say that for startups, but for scales up to, you know, don't worry too much about, like, a one source of truth. [00:37:32] That is such an oasis, like a myth, a utopia. There is, there's never gonna be like a one source of truth, where you're gonna be able to la la la la la see all your channels, all your touch points, and be able to, you know, perfectly attribute those results. So, don't worry about like an MTA or um, a unified reporting, just try to measure the activities that you're doing.[00:38:00] [00:38:00] And if you're a startup, you don't have the data team resources. If you're B2C, you know, have like a user option on how did they find out about us. And if you're like B2B, ask Lee. I always find, you know, good leads questions are, you know, how did you find out about us and what, what problem do you think we can help you with? [00:38:24] And how do you think we help you solve? Because that kind of tells you, you know, one, you know, the attribution, but it also tells you the positioning that is working. So just make sure to, you know, gather that data and store it. And I think if you have these things, you already have a very good. intuition, but it's not. [00:38:47] It's just qualitative data on what is working and, and not working. And then you can take it from there. [00:38:54] Phil: Yeah, I think it's great advice. [00:38:57] The Value and Limitations of Self-Reported Attribution --- [00:38:57] Phil: Uh, I'm curious to ask you on like the, the self reported [00:39:00] attribution question there, like it is very polarized. Like some people love it and they put a lot of green and like focus on it. And some people just like, Shame it and they think that it's completely invalid data because humans aren't perfect. [00:39:16] Humans have horrible memory. Humans are irrational, predictably. And if you ask someone a question of like, how did you hear about us? What are the odds that this person remembers that the first Actual impression was a display ad on like an unrelated website or it was a banner they saw when they were driving and maybe they give credit to something that was like actually in the middle of their journey. [00:39:39] Just curious your take there. [00:39:41] Barbara: Yeah, I, I, I think of course, you know, but you can say the same about the customer interviews. You know, why am I going to interview customers if they're going to give me this biased view? Obviously you take everything they say with, uh, with a heavy grain of salt. I would say, you know, it's not necessarily [00:40:00] the perfect attribution, but okay, if they remember a certain touch point that tells you something about that touch point too, right? [00:40:10] Phil: Okay. Yeah. That's super interesting. [00:40:13] The Pitfalls of Overemphasizing Attribution Accuracy --- [00:40:13] Phil: Um, so you mentioned like there, there is no oasis of like a source of truth for measurement. Um, so let's say like. I'm a listener and I'm listening to this conversation and when I go to these like attribution meetings, we have one person showing this set of data. [00:40:35] Yeah, and like this other person on a different team is saying, no, like the credit should be given to this and like all of these different methodologies, opinions, ideas, and that meeting, like, How do you make sense of all of that? Like if there isn't an oasis of a source of truth, you said your advice to startups is to just like measure one thing and just kind of go with it. [00:40:55] Like how much effort do companies put on figuring out [00:41:00] the trust behind the measurement methodology versus just doing the thing and like getting stuff out the door and like drive in revenue. Like I've struggled with that like my entire career. I curious your advice. [00:41:12] Barbara: Yeah, they put way, way too much focus on these things, don't they? I would say, yeah, one of the things that, um, uh, Happens a lot is regarding incentives, right? When you have, you know, yeah, different teams that want to receive the attribution credit so they can sometimes, you know, employees even have bonus attached to it. [00:41:36] You know, if you bring in, you know, an X number of, you know, conversions, then your bonus is X. To me, that's all like a recipe for disaster and for wasting everyone's time on trying to, trying to figure these things out. [00:41:53] Phil: those are never fun conversations, especially like, but they have been less in my career later on, but [00:42:00] like early on [00:42:00] it was always like marketing fighting sales and like, was it marketing that influenced that pipeline number? And anyways, yeah, it's, uh, it's, I think like one of the least fun Productive parts of attribution when it comes to just like what team should be creating credit for something like we're all part of the same company. [00:42:21] We all have the same goal. We're all trying to grow the company. Why are we spending meetings arguing about who drove this and who drove what? So, yeah, I like. [00:42:30] Barbara: I mean, there's definitely times that I say that it's better to have no data than data. Because if you don't know how to interpret or how to use it, that's actually more damaging to your company than just going along. I was having a conversation recently with someone from Uber, and they were saying that the way, you know, Uber calculates CAC is you, they just take all costs. [00:42:56] And divide by all new accounts. And that's, and [00:43:00] that's their calculation. And I was like, that is perfect. [00:43:04] Phil: interesting. As long as they're keeping that definition steady over the years. Um, yeah, but [00:43:10] Rethinking Testing Culture in Marketing --- [00:43:10] Phil: I'm curious to ask you about testing culture because like, I think, you know, there's this, this, I forget like who said this, but like there's most tests, like 70 to 80 percent of tests, probably more fail to show incremental value because they don't get statistical significance, but Well, that's kind of part of learning, right? [00:43:29] Like you're trying something and you're hoping that it leads to something and it doesn't always have detectable effect, but I think most of us understand that testing, like should always be linked to specific questions or goals. And I've asked this to a lot of guests, but like, does everything need to be a test is testing culture dependent on this? [00:43:49] Like as qualitative data enough, sometimes curious, your take. It [00:43:55] Barbara: it depends. I feel like it depends, it's just always an [00:44:00] answer that comes with me. I, I don't think that you can do attribution without a testing roadmap. Because even something like MMM, it needs to have variations. I to it in order to calculate casualty. So, you know, that's increasing budget that is turning off in certain regions. [00:44:22] That is, that is very important. I think there's a lot of companies that do tests without a methodology first. And that is, you will be surprised by how common that is. They're like, Oh, we're going to test this. And then later, how do we measure? And yeah, that's probably the reason why 80 percent of tests. [00:44:44] don't, uh, don't drive any results because they weren't set up with a hypothesis and with a methodology before they, they started running. But I think like everything, everything that you [00:45:00] do in marketing, in measurement, you only have an X number of hours in a day. So you have to make sure that what you're focusing on is what can actually drive an impact And indeed, I think, you know, yeah, testing, testing buttons, testing landing page changes, testing certain, yeah, a lot of the times it's people want to postpone decisions, I feel, or they want to take away that, that responsibility. [00:45:36] Phil: Yeah. Subtle arguments. [00:45:40] Barbara: and, and, and I always also say, you know, like, You know, marketing data, none of it is, is real. These are all estimates. we just have to also remember that. [00:45:53] Phil: I like that. That, that might be the teaser for this episode, but all marketing data is real. [00:46:00] It's all estimates. [00:46:02] Barbara: no, it's [00:46:03] Phil: of marketing [00:46:04] data. [00:46:04] Barbara: it's it's true, you know, because every single metric you look at, It's not the real number, you know, users, it's not, it can be devices, it can be multiple users in the same household, page views, you know, there's bots, there is, you know, um, cookie blocking, there is, there's different consent laws. [00:46:26] None of these numbers are, are, are real. Yeah. [00:46:32] Phil: Yeah, a real or a hundred percent accurate. Well, maybe we can, uh, I got a couple last questions for you. [00:46:39] Building Strong Foundations for Effective Marketing Data --- [00:46:39] Phil: So like maybe on this topic of like data, data quality, like there's, there's, um, there one guest shared like three components to effective marketing data. Like before we even get to. Attribution and measurement analysis, whatever. [00:46:55] Like there's foundational data management component to all of this. Like, even though [00:47:00] we know like there isn't this perfect, like data with marketing, there are some things we can do to at least like get to a number that we're, we're confident with. Um, but I'll give you the list of all three that, um, folks have mentioned are kind of essential for effective marketing data. [00:47:16] I'm curious if you'd add anything or if you agree with them. So the first one is ensuring that. You're gathering relevant data, like user interaction with ads. Um, the second is categorizing the data consistently across platforms to avoid misreporting or limits and misreporting as much as possible. And the third one is connect that data both within marketing channels and with internal company data using join keys and a bunch of different tooling there. [00:47:46] Would you add anything curious if you agree? Yeah, [00:47:58] Barbara: I think, you know, especially when it [00:48:00] comes to paid media, there's only so much value you can get based on what's available out of box on ad platform reporting. So all contacts that is relevant for your product should go into naming conventions. [00:48:15] I would add, and I don't know if that would fall within the scope, but literacy. And education. It's not enough to just have the data. You need people to be able to use the data and understand the data. Um, I would add, now I'm just thinking, I'm sorry, Phil, I didn't prepare that much for this question. So I'm just trying to, but I think that there's all of these, you know, best practices that are ignored. [00:48:44] I would ask, I would add testing roadmaps. I think, you know, making sure that that's included in your, in your daily plan and there's a cadence to that is very important. Hypotheses [00:49:00] are very important, making sure that you know what you're trying to measure. I think strategy based attribution, like I said, is also very important. [00:49:12] That you're not just trying to get this, you know, macro view to allocate budget, but you're also really trying to look at each strategy separately and understand what should this strategy in specific measure. You know, like a top of funnel campaign is very different from a bottom funnel campaign. So because of that, you need to have a different way of measurement. And I think These are some that I would add. Accessibility, I would also add, like what you were saying before, you know, back in the beginning of the conversation that, you know, there's a lot of data warehouses, but marketers are excluded from it. Domain experts need to be able to model, need to be able to transform the data, have access to [00:50:00] it, instead of have everything be delivered in this dashboard that they go on to download and use Google Sheets, you know? [00:50:10] Phil: go on to download and not understand behind the scenes, how the data is powered and where it's getting pulled from. No, this is all great. Really great advice. [00:50:19] I especially like the, the nomenclature piece, but also the like understanding experimentation design. Like, I feel like a lot of people in marketing are just like, yeah, yeah. [00:50:30] Bridging the Gap Between Data and Marketing Teams --- [00:50:30] Phil: I'm really good with experimentation, like hypothesis, get it out. Like run it for as long as you need to like, uh, 90 percent confidence, whatever. But I, I had a short stint at, um, my only real big company enterprise experience was with, uh, automatic wordpress. com was taking a lot of a fire right now, I don't know if you're, you're paying attention to, uh, yeah, it's pretty cool being a, an external person now, uh, keeping tabs on all that [00:51:00] drama. [00:51:00] But one thing they did super well at wordpress. com is they. Had a big team of data and one, uh, a lot of internal tooling was built and we had an internal experimentation platform and it was cross channel and it was run by a team of data scientists and data engineers and part of their role was just. [00:51:23] Experimentation design literacy within like the entire marketing org. And you couldn't create an experiment and the platform until you had gone through the training on like how to use the tool, but also. What is the minimal detectable effect? What is the length of your experimentation? What is a good hypothesis? [00:51:46] Like all of those things were just like prerequisites. And I was like one of those marketers. Yeah. Yeah. Like I'm, I'm good with experimentation when I finished that course and started using the tool, like it was just like. Opening up to a [00:52:00] world where like data scientists were actually doing the teaching for marketers. [00:52:04] And yeah, I really like that, that, that point of device there. [00:52:08] Barbara: Yeah, no, that's great. I, I think, you know, if I could have, give one takeaway to companies in this podcast, it would be, you know, try to get your data and your marketing teams to work a bit closer together because they both need each other. You know, data teams are struggling to show the business value they provide and, and marketing teams are struggling. [00:52:32] To make decisions based on data so they can help each other out a lot [00:52:36] Phil: Yeah. Yeah. A hundred percent. I think that that is maybe one of the ethos of, of the podcast. Like a lot of the listeners are either on. The one end of the spectrum, like they're on data teams or they're more on the creative side of marketing, but they're coming together in a more like pseudo technical marketing role. [00:52:55] Like a lot of folks have called it like a data product manager or martech [00:53:00] product manager who works in between both teams. And they're like translating stuff, right? Cause like did engineers don't have time to learn the. Life cycle marketing use cases of why we do Dunning campaigns and abandoned card campaigns and marketers don't always have the bandwidth to learn everything about experimentation design. [00:53:18] And so, yeah, like that, that bridge in the gap between the two, it's, uh, what, what makes the show really awesome. And, uh, I think the insights you brought today are super helpful to, to, to spur that mission there. So. Really appreciate your time. [00:53:30] Finding Balance in a Goal-Oriented Career --- [00:53:30] Phil: I got one last question for you, Barbara. Um, you're an analytics consultant, obviously you work with awesome brands. [00:53:36] You're also creating a ton of valuable content. You're also a big tennis player. You're a home chef and a foodie. You're also a dog mom. But one question we ask everyone on the show is how do you remain happy and successful in your career? And how do you find balance between all the things you're working on while staying happy? [00:53:53] Barbara: a great question. I'm very much of a goal oriented person I really like to [00:54:00] also gamify things So I really like targets and that's probably the lamest answer You've gotten on this podcast [00:54:09] Phil: It makes sense. A measurement expert [00:54:10] Barbara: I, I, I adore targets and hitting targets and targets that don't mean anything to anyone else but me. And yeah, I, I, uh, I've always, I guess, had a pretty relaxed life. [00:54:26] You know, I try to take a lot of time off throughout the week. I went to the market earlier today, you know, to buy some nice food for me, some nice treats. Went to the park with my dog. So I try to not think of work as, as something that, you know, just needs to go on for the whole day. And I try to work really, when, when I feel like it, I guess that sounds weird, [00:54:53] Phil: no, no, it makes [00:54:54] Barbara: you know, there are times, yeah, that I, I don't really feel like working. [00:54:58] And then [00:55:00] that's fine because that's just life. [00:55:04] Phil: Yeah. Yeah. You have your flow state. You're way more productive sometimes when it's dark outside. You're a bit more of like a nocturnal owl and definitely like that myself. So. Yeah, super good advice there, Barbara. This has been a super fun conversation. There's probably a bunch of other jumping off points. [00:55:21] Um, I'll link out to, um, the upcoming course that you're doing for the next cohort, uh, but your newsletter as well, I think a great resource for, for marketers and data teams, um, uh, as well. So yeah, really appreciate your time, Barbara. It's just super fun. [00:55:35] Barbara: Yeah. Perfect. Yeah. Thank you so much for having me. So this was great.