Subscribe
Share
Share
Embed
Dave Gerhardt (Founder of Exit Five, former CMO) and guests help you grow your career in B2B marketing. Episodes include conversations with CMOs, marketing leaders, and subject matter experts across all aspects of modern B2B marketing: planning, strategy, operations, ABM, demand gen., product marketing, brand, content, social media, and more. Join 3,500+ members in our private community at exitfive.com.
Pranav Piyush [00:00:00]:
1234. Exit.
Dave Gerhardt [00:00:02]:
Flower.
Pranav Piyush [00:00:08]:
Exit. Exit.
Dave Gerhardt [00:00:15]:
So without further ado, Pranav, I'm going to turn it over to you. Let's kick things off. All right, here we go.
Pranav Piyush [00:00:20]:
Thank you, sir. Hello. Hello, everyone. This is amazing to see of you from everywhere around the globe. This is going to be a fun session. So as Dave mentioned, this is not going to be a boring webinar. This is going to be an interactive session. I'm going to make sure of that.
Dave Gerhardt [00:00:36]:
That's because it's not a webinar. You said.
Pranav Piyush [00:00:42]:
The top, and I heard from Dave. Right. This is a popular topic around how to measure your marketing efforts. This is the evergreen topic. Everyone's always talking about it. And I'm particularly excited because I'm going to tell you, it is not that hard. It is not that hard. This is something that you can achieve, like today and tomorrow, and make it really practical.
Pranav Piyush [00:01:05]:
And we're going to talk about that stuff today. So that's the quick sort of intro to the session. I'll tell you a little bit about myself. So who am I? I have spent the last 15 years working at companies like PayPal, Dropbox. You can see the logos here all across, strategy, product growth, and then marketing. So I was one of those weirdos who thought, oh, yeah, marketing, that's easy. I can do that. I raised my hand and said, yeah, let's go figure out how to do marketing.
Pranav Piyush [00:01:32]:
And then really learned how hard it is, the combination of the creative and the art, and then being able to stand in front of your executives and your board and talk about what business did you drive for your company and realize that that's hard and we can make it easy. And that's what I'm here to do for all of you, hopefully. So, based in Burlingham, California, if you're in the know, say hi, we'll catch up over coffee and what have you. So the first thing that I like to talk about when we're talking about marketing measurement is investor first, marketer second. So what does this mean? All of us, when we are thinking about marketing measurement, are thinking about, this is what I can do for the business. This is what I have done for the business. This is what my team has done for the business. We deserve more.
Pranav Piyush [00:02:25]:
We deserve more credit. We should be celebrated more. But you have to take a giant step back and think about what is the goal of the business and the people who are funding that business, whether it's bootstrapped, whether it's VC backed, whether it's a public company at the end of the day, there are shareholders and investors and board members and executives who think about things like return on investment. And you have to think about investment first, then investor first mentality before you think about all the amazing creative ideas you have as a marketer. And that's a very important starting point because it reframes how you think about measurement. So that's step number one is think about and put on your investor hat. And as an investor, if you wouldn't invest in a certain business strategy, whether it's marketing or sales or product or a distribution method, then everything else doesn't really matter, right? That's step number one. Yeah.
Dave Gerhardt [00:03:27]:
What you said is this is also step number one. In just like there's discussions come up a lot in exit five or just beyond a lot of the audience here are folks that want to or are move into a marketing leadership role. And oftentimes we'll ask you what's the difference between individual marketer or maybe like small team manager into marketing leader? And I think it's this. Marketing is the thing that you own, but you are responsible for the business. You have to understand the business strategy first. And I feel like at least in my career, anytime I felt misaligned, it was because I didn't fully understand the company strategy. When I was really tight with the CEO, my life as a marketer was much, much easier. It just removed all the bullshit and helped me focus on what matters.
Dave Gerhardt [00:04:12]:
So I love this frame of investor first, marketer second.
Pranav Piyush [00:04:15]:
Nice. I completely agree. And now, don't get me wrong, there are many companies where the strategy might be messed up or nonexistent. So I'm not saying that you're going to just get it right if you think about investment first. But if that is the case, you need to call it out or go find a different job where the strategy is sounder, where you can have an actual impact with marketing. So that's step number one in my mind. Step number two, and this is something that I will talk about a lot. We use these words, right? So justify investments, right? Marketing sourced, marketing, influence, marketing attributed.
Pranav Piyush [00:04:49]:
We use these words. But if you take a giant step back and think about what are you trying to achieve with marketing, it's going from a smaller business to a bigger business. Pretty simple. And at the heart of that conversation is this word called incrementality. This might be new to some of you, this might be new to many of you. It was new to me a few years ago when I first sort of encountered this word. And what this means is, are you adding incremental net new business for that company and not just fighting over credit? Because that is not important if the business isn't growing, if your new sales aren't growing, if your new leads aren't growing, if your new traffic isn't growing. And that's another sort of important reframe.
Pranav Piyush [00:05:39]:
It's not about the credit and the attribution. What it is about is are you adding net new on top? That would not have shown up at the business if you weren't doing your job. And if you reframe it that way, it all comes full circle to that investor mentality. I'm going to pause and see Dave, if you have any thoughts on this word. I know incremental can sometimes be perceived as this, like, oh, it's just know doing small things, but that's not really the meaning of the word. It's incremental means that it's on top of the baseline, on top of what's already coming in inbound.
Dave Gerhardt [00:06:11]:
Yeah, I think it's easy to just get caught up in trying to have the perfect end to end solution for measurement for marketing strategy. I love this idea, and some of the most fun that I've had in marketing is at a smaller company when you can shrink things down to what are we working on this week? I think it gets hard as you stretch those things out. But I love the idea of incrementality and just thinking of like, look, we're trying to get here. We don't have all of the data, we don't have perfect answers, but every day, every week, every quarter, we're going to get closer and look at what we'll be able to do now. I think about HubSpot a lot as an example, only because I happen to work there early in my career and to see where they are now. I just recently did a podcast with Kip, who's the CMO. It's like every problem or challenge they had, like seven years ago when I worked there, they've solved or iterated on. And these things just take time when the longer you have smarter people working on the business.
Dave Gerhardt [00:07:10]:
So I love this approach also, anytime you can throw Mario in the deck. I spent $150 on ebay to get my kids a Nintendo 64 so we could play Mario Kart. Nice. So well done.
Pranav Piyush [00:07:22]:
Yeah, I got my seven year old the Nintendo Switch, so she's all about.
Dave Gerhardt [00:07:27]:
Exploring the just to build on that. So I went super early days OG. So I said, we can get a Nintendo 64. We went to my cousin's house, they had Mario Kart and the switch. And my son looked at me, he's like, dad, what is that they're playing? I'm like, it's the same game, just 30 years of graphic advancement.
Pranav Piyush [00:07:47]:
Yeah. Hey, don't change what's not broken, right? Okay, so when we talk about this concept of incrementality getting more specific, what does it mean? What is incremental? And I boiled it down to this graphic, and this is a really important one, and hopefully it'll help you as you think about sort of your marketing measurement challenges. Think of it this way. If a prospect would have become a buyer, regardless of what marketing that you did, then that marketing isn't really incremental. And vice versa. If a prospect would have never become a buyer until and unless they saw a piece of marketing, then that piece of marketing is super incremental. That is the heart of the marketing measurement problem. And the question that your CEO, that your board is trying to answer is that campaign, is that channel bringing in incremental net new leads or sales that would not have come to us anyway? And if you think about it that way, everything else will figure itself out.
Pranav Piyush [00:08:51]:
Now, I will add something here. It doesn't matter how many clicks you get, how many impressions you get, how many people say what on your self reporting forms, all of those. I get what you're trying to do. At the end of the day, the discussion is, would those people have come to you anyway or not? And if you can answer that, that is the holy grail. So again, I'm going to pause and see Dave, if you have any thoughts on this.
Dave Gerhardt [00:09:12]:
I just screenshot of this and I sent it to our team internally because we're trying to figure out how to repeatably grow exit five right now. And we've just hired a team, we've just invested and we're trying to start new channels. And we have this baseline of how many people show up to our website and start a trial when we don't really do anything. Now we're doing stuff like we're writing on LinkedIn, we have a newsletter, but that's kind of all kind of steady state stuff right now. We're trying to figure out like how do we go from 200 trials a month to 225 to 250 to 300? I literally just screenshotted it and I'm typing up this note as I multitask here because I'm like, yes, this is how we figure out what we need to go and do next. Exactly your baseline and aren't there those classic, like, I think it was eBay was one and Airbnb was another, where they turned off paid search and all the same, people still showed up anyway. Is that kind of what you're what? Exactly. Okay, yeah.
Pranav Piyush [00:10:12]:
If you Google eBay paid search experiment, this was, I think, like 2010 or something like way back or might have even been earlier than that. They were the first ones to run this giant experiment of let's stop paying Google money for a little bit and see what that does. And it turns out everybody was going to eBay anyway. Those ads were just siphoning off credit at the bottom of the funnel. And so you really have to think about the incremental impact of every channel and strategy. And it's hard, right. It's not easy to get this answer. We're going to walk through how you can do it at a small budget and a large budget, but it's critical to at least know sort of what it is that you're aiming for.
Pranav Piyush [00:10:49]:
Okay, now, I will also say this, right, so we talk to marketers all the time at all stages of growth. And the six questions that always come up are these six questions. There isn't some, like, you don't need to answer these 500 questions to get marketing measurement right. It's these six. If you can answer these six for your CFO or your CEO, you're done. You're going to get promoted. No questions asked. If you can answer these questions.
Pranav Piyush [00:11:17]:
So think about that, and everything rests on the first question of incrementality. If you don't get that question right, none of the other math actually makes sense. So remember that. Now, I know we had a little bit of a clickbaity. I don't know, maybe subheader on our non webinar. That said, click attribution is dead. What do I mean by that? And why do I say that? And I have nothing against tracking clicks, measuring clicks, just like I have nothing against tracking self reported attribution or measuring self reported attribution. You have to, however, go into the real depth of what you are capturing with that tracking and what you're missing.
Pranav Piyush [00:12:00]:
And these are the four points that you should just be aware of. Number one, just by the virtue of the fact that you're tracking clicks, you're going to be biased against many channels, video, social, whether it's organic or paid. Obviously, anything that's happening in the real world, right? Connected tv, YouTube, billboards, direct mail, none of those are going to get captured in a click based attribution model. The second is all the offline marketing, even things like events, billboards. Well, obviously you're not going to be able to capture that in clicks, right? And then all the changes that are happening in the privacy side of the house. I discovered this, Dave, you'll find this funny. Apple in the summer of last year, started to strip out UTM codes from Apple Mail and Apple messages. And I'm like, what? Yeah, that's happening right now.
Pranav Piyush [00:12:49]:
And every year some new sort of privacy change that comes about, especially from the likes of Apple, that makes it harder to use click tracking the same way that you've been doing for the last 1020 years.
Dave Gerhardt [00:13:00]:
Steve Jobs would be rolling in his grave if he knew about utms because I forget which book it was that I was reading. Maybe it was this book called insanely simple, but if you've ever seen the URLs for launches that they do, it's like Apple iPad or Apple Pro, we have ruined that. I love the vanity URL. Let's do better on our end. To drive to a like, we put so much like, okay, you got to have this UTM because if somebody didn't click through this link, it didn't happen. Well, guess what? I delete all those utms out all the time. Because I just think the clean, simple. Look, Dan and I, we had this argument a couple of weeks ago.
Dave Gerhardt [00:13:44]:
He's like, yeah, put this UTM in your twitter bio. And I was like, no, that looks awful. It looks like exit five, question mark.
Pranav Piyush [00:13:54]:
It should just be clean.
Dave Gerhardt [00:13:55]:
It should just be exit five.com and we should figure out how to measure it from there. That's at least the dream, right?
Pranav Piyush [00:14:00]:
Yes, I think you're 100% right. We did something similar. We're fortunate that we're a young company and we can make these decisions right now. But we decided no UTM codes ever. No cookies on our website ever. No click ids. No, none of that on our website and our business ever. And I think it's very possible there's other businesses who are doing very similar things and we can happy to chat about that with other folks.
Pranav Piyush [00:14:25]:
The last piece that I will add here in terms of clicks is to understand incrementality. You're really trying to understand causation, right? What that means is I did this thing in the real world in terms of marketing and it caused my business to grow. And just by tracking the stream of events or clicks, you don't have a sense of causation. Forget about even correlation. That's also not happening. It's what I call coincidence that those clicks happened. And that's the thing that you have to remember now. It doesn't mean don't track these things, it just means don't use them to answer the question of ROI and incrementality.
Pranav Piyush [00:15:06]:
Okay, so lots of build up. So what is one supposed to do, right? I'm painting this really grim picture. What do you do? How do you make this work? The first thing that I will say is if you are spending a few hundred thousand dollars a year, if you're less than a million dollars in marketing spent across paid, owned and earned, and if you're primarily working with one channel or maybe two channels, I would argue you don't need much other than a spreadsheet. This is true for our business, right? I have a spreadsheet that tracks the key sort of fundamentals of our business, which are things like how many people showed up to our website, you don't need any fancy tracking for that. Everyone has that. How many people raised their hand on our website and said, give me a demo. Very easy to track. I can look at that over time.
Pranav Piyush [00:15:58]:
I can see my organic social activity, that's my only channel today, right? Organic social and community. And I can see how many impressions I'm getting on every single post. The correlation will hit you in the face with a simple spreadsheet like that. So that's number one. If you're a small business, if you're a marketing team of one and you want to get this up and running in just a simple spreadsheet to simplify your like shoot me a note, happy to show you how we do it and help you out. But as you grow and as you get multiple millions of dollars in budget and now you have multiple channels, you've got LinkedIn going, you've got meta going, you've got events going, you've got email going, you've got Google search going, then it becomes really hard to understand which of those is really causing your sales to go up or not. So the first thing that I say in this process, I've got five steps. So we're going to walk through each of these steps one by one and I'll be quick.
Pranav Piyush [00:16:55]:
First step, you've got to model out your business. What do I mean? Here we all talk about, and one word that you will not see on the slide is leads. Because I think that word has been tainted. It's a perfectly good word. There's nothing wrong with leads, but I will not use it because everyone will shit on me. If I were to use it. Sorry for the French.
Dave Gerhardt [00:17:15]:
Okay. I love leads. Let it be known. Put it on the record. I love leads.
Pranav Piyush [00:17:19]:
Yes. Great. I'm calling it a hand raiser. And I think that that was the true definition of lead. Right. What this is saying is give me a demo, give me your pricing, let me talk to sales, let me in. I'm raising my hand and telling you I'm interested in engaging with your product and getting into a buying process. And so if you start with that, that is a leading indicator and you want to really understand what that is looking like for your business over the last twelve months and what the forecast for the next twelve months is going to look like.
Pranav Piyush [00:17:53]:
If your sales cycles are long, you got nine months. Twelve months sales cycles. We deal with that on our business. You don't have twelve months to see the attribution on a deal before you decide whether that piece of marketing is working or not, you need to have a leading indicator. And so model your business. Understand the variables in your business model and which ones you're going to optimize towards. And you want to pick metrics that are aligned with your customer journey, not how you like to think about things in terms of scoring and quality and qualified. All of that jargon is not necessary.
Pranav Piyush [00:18:29]:
So that's number one. Model your business. Understand the key inputs that marketing can truly impact. Before you think about how do you measure the impact on those metrics?
Dave Gerhardt [00:18:40]:
Yeah, it's great. I think this is an important step and it's easy to just get sent in a bunch of different directions, trying to figure out what to measure when not everything matters and not everything holds equal weight. And you do kind of have these different paths that you're trying to push people towards, whether it's get a demo, contact sales, start a trial, sign up for the free version. And so I think getting everybody in agreement on what is the buying journey, what is the buyer's journey here, then we can layer on how we're going to do marketing, where sales is going to come in and how we're going to measure this.
Pranav Piyush [00:19:11]:
Nice. Exactly. And I will say something right, this idea that marketing can somehow influence every part of the journey is just completely wrong. Once you get somebody in the funnel and they're an opportunity to go from opportunity to a deal, the amount of impact that marketing can have on that is minuscule compared to the sales experience, the solutions experience, the pricing, the actual product, if you have a pilot process and all that stuff. And so you might actually be wasting your effort trying to get deal acceleration or pipeline acceleration, if the ability of marketing to drive that is nonexistent. So just think about that and be really intellectually honest about what you can actually move for the business. Okay? Second, measuring what matters and understanding the correlation between inputs and outputs. So how this works.
Pranav Piyush [00:20:02]:
And like I said, you can do this in a spreadsheet. And if you don't have a business analyst or somebody who understands data, who understands correlation, then I recommend two things. Either go hire somebody or go find a vendor who can help you do this type of math. But once you've modeled your business and you've said, okay, we're going to try and increase hand raisers, because that is the leading indicator, that is the demos, the trial starts, all that fun stuff that we want to sort of grow for the business. You're going to look at every single marketing channel, whether it's paid, whether it's earned, whether it's owned. So pr, direct mail, events, organic, social, paid, social, paid, search, everything, and you're going to map out the last twelve months or more of reach and impressions across every single channel. You're making them apples to apples. That's number one.
Pranav Piyush [00:20:57]:
And that's why we're picking impressions. And you're going to see, when my reach on Facebook goes up, when my impressions on Facebook go up, do I see a corresponding increase in my demos or not? When my impressions on Facebook go down, do I see a corresponding decrease in my demos or not? Very basic, very simple, something that even a ten year old should be able to understand. And when you do this over a twelve month time period, over a 24 month time period, you will figure out the correlations between each marketing channel and campaign and your demo volumes. That is the idea behind understanding the correlation between all your marketing channels and your outputs. And what usually will happen is you'll get an output back from this. That shows you, oh, Facebook seems to be very highly correlated. Every time you spend more and have more impressions on Facebook, your demos do go up. Or you might see that Google is not correlated at all.
Pranav Piyush [00:21:59]:
This is a hypothetical example. I have nothing against Google, I love them. But just showing you a potential way of looking at the business. Okay, so what does that do for you? What is next? When you pull that model together, whether it's a spreadsheet, whether you hire a vendor, whether you have a data science team doing this for you, it outputs something like this. You literally see incremental versus baseline as an output of that model. And I'll go back to the question that we started with around this question of incrementality, right? So you're using statistics on top of all the things that you measured, and you're now seeing your overall impact of all your marketing investments. How much incremental business did that drive versus what was baseline, which is your organic demand, your word of mouth, your built up brand equity. So going back to Dave that you were saying about exit five, you already have a baseline of whatever, 200 trials a day or a week.
Pranav Piyush [00:22:58]:
Day would be nice, wouldn't it? And how you go from 200 to 25, this is what you're going to get out of that analysis. All right, my favorite statement, correlation is not causation. So let me explain what I mean here. Just because a couple of trend lines move in the same direction doesn't mean that there is a causal relationship between those two trend lines. And it is possible that you said in a previous example, you know, every time we invest more in know, your demos are also going up. But maybe there's something missing in the data that just at the same time when you go up in Facebook impressions, you're also doing events and somehow that got missed in the data set. And so you're finding a correlation where there may not be any. And so this is the next step in marketing measurement, which is causality.
Pranav Piyush [00:23:52]:
And this is all about experimentation. And it might seem, well, we experiment all the time, don't we, in marketing? And I'm here to tell you that you probably are not running experiments the way you should be running experiments. So there are three ways in which you can run experiments in marketing. One is pretty much, there are only about three platforms that offer this capability natively. There's meta, there's YouTube, and I think TikTok is getting into it. And this is called a conversion lift test methodology, where when you go into Facebook and go into the experiments tab, or the same thing in YouTube, you can set up and tell Facebook to split your entire audience into a control and a test bucket. And they will then ask you to upload your hashed conversion data to Facebook. And then they will tell you that when you serve your ads just to the test bucket, what is the incremental impact of that spend? And you're not using clicks and touches to inform that measurement.
Pranav Piyush [00:25:01]:
You're using your first party data that you're feeding through an API back into Facebook. So I'm fairly sure that a lot of folks have not heard about that testing capability. It is not the same as a b tests that are shown in the UI. This is something that is a lot more hidden. And I can explain why it's hidden in Facebook, but that is the first option that you have. Now, what about a channel where you don't have a native conversion lift capability? The second method is actually much simpler, geotesting. What is geotesting? It's literally turn your advertisements or whatever marketing strategy that you have, turn it on in a couple of regions and turn it off everywhere else. Pretty simple, right? So I'll give you an example.
Pranav Piyush [00:25:46]:
If I wanted to turn on LinkedIn advertising tomorrow for Paramark, that's what I'm going to do. I'm going to say I'm going to turn on advertising in California and New York, and I'm going to turn it off everywhere else in the US. And now I'm going to track my traffic, I'm going to track my leads, I'm going to track my demo signups from those two regions and everywhere else in the US. No special software needed. You can do this yourself if you are at a low volume of spend and you can start to see a pattern immediately and go back to your CEO and CFO. Hey, look what happens when I turn on advertising in these couple of states and it's off. And everywhere else you have irrefutable evidence about your lead volume, your traffic volume. The easiest way of proving the impact of what you're working on, no software needed.
Dave Gerhardt [00:26:33]:
One time I was at a company and we raised a bunch of money and we did a huge press announcement. We were featured in all the top publications and that day was the biggest spike in inbound interest in the company ever. And I hated that because for the rest of my life, the rest of my time at the company, we always looked back at that spike. And whenever someone would be like, what did we do that day? And I was like, well, we've raised $65 million. Hey, look, if you want to do that once a quarter and give me that, I'm happy to make a big announcement and a big splash, but I wish I could say this is repeatable, but it's not.
Pranav Piyush [00:27:12]:
Yeah, that's exactly it. And I am very confident now, having been doing this for a few years, that every single marketing channel and strategy can be tested with a control and test sort of mentality. And you don't need a ton of support to be able to make that happen. Now, if you're spending hundreds of millions of dollars, billions of dollars, like some of our clients do, then, yeah, you're going to need a little bit more help because now you're running 50 tests at the same time. And you're slicing and dicing the data by DMA and geography in many ways around the globe. And it gets complicated. But if you're starting off, this is the simplest thing that you can do. I was talking to a marketing leader at a seed state startup just a few days ago and she was asking me about, hey, how should I think about growing our non brand search ad spend in a profitable way? This is what I told her.
Pranav Piyush [00:28:04]:
Whatever new concepts you have, run it in just one city and see how that works for that one city. Do you get more traffic? Do you get more leads? And if you do, you can scale that up to as many geographies as you need to. And that gives you a repeatable process to test and learn any new strategy. Now, there is one other methodology. Even if you don't do geotesting, you can actually do a time based test. And this is in layman's terms, you turn something on and you turn something off again. That's another way for you to get data. So I'll give you an example.
Pranav Piyush [00:28:37]:
I actually ran this test for our own business in December. My posting frequency on organic social was like half of what it was in Jan. And I improved the frequency and I changed a couple of things about what I was posting about. And I can tell you with extreme certainty, looking at my data, that the spike that I saw in leads in January versus December was a direct result of that change in intensity and change in tone. And so you can show that with evidence to your leaders if you know what you're doing and how you're constructing that test. Okay, step number five. This is the last one. So you've modeled the business.
Pranav Piyush [00:29:20]:
You have measured what matters. You've estimated incrementality using correlation. You have now run experiments to understand the causal impact between your marketing investments and your business outcomes. The last step is probably the most important step when you're talking to execs, and that is forecasting, at the end of the day, what they care about, what your head of sales cares about, what your head of marketing cares about, what your CEO cares about, what your board cares about, where are we going to be in the next quarter? And everything that you've done up until now is helping you inform this answer. And what I will also tell you is that they're not looking for this precise number of you must hit that number. That is not how cfos think. That is not how boards think. They're actually looking for your best case and your worst case scenarios, and they're going to do risk management.
Pranav Piyush [00:30:13]:
On top of that, when they report out numbers to the street, if they're a public company, or even if they're reporting out numbers to their board, they're never presenting one number. Never. So when you understand your historical numbers over the last twelve months and you have this model set up, you can actually, the model will predict where you will be in the next quarter with all of your marketing spend accounted for. And that will allow you to then generate a forecast and you go in with a range of this is the best case, this is the worst case, and here are all the experiments that are in play to try and beat the best case and go even higher. And that's the communication sort of, I would say skill that you need to develop to be able to talk to your boards and your execs and your finance team in a smart way. That's what they're expecting. All of the stuff that I talked to you about today can be done with Excel. It can be done through open source data science models, it can be done through vendors.
Pranav Piyush [00:31:15]:
There's many of them. The industry term for these things is marketing mix modeling, incrementality testing. You can put that into Google and you'll see a whole bunch of vendors. Obviously Paramark is one of them, so check us out. But that's something that I would recommend. Anybody who's spending more than a million, 2 million, you're starting to get to that scale. This is the way to run your business. And it's not just me saying this.
Pranav Piyush [00:31:44]:
We've talked to marketers at companies like Asana, DoorDash, even PNG and Unilever, indeed square. These companies are all using this type of methodology across b to b and b to c, and there's no reason you can't do this yourself. So that's the meat of the topic today. Dave, any questions, reactions at this point before we go into our special session? Right after this?
Dave Gerhardt [00:32:13]:
No, this is think for me, I think this was helpful because I think more than saying like, hey, use this software, use this tool. What you presented was a fundamental approach to measurement, where you have to be able to think about your business and understand where measurement fits in, as opposed to just being like, let's start measuring everything. This is very much driven in how does our business operate? What matters? I will say one thing that I wish I was better at earlier in my career was the forecasting piece of it, because I was so focused on the short term, like, what are we doing this month, this quarter? The ability to be able to measurement to me, isn't just important because you need to justify spend. That's one way to think about it. Measurement becomes really important when you say, okay, we did a million dollars in revenue this year. We want to go to $3 million next year. Where is that going to come from? That's where measurement becomes really important because you need to know what you did, what the numbers look like, how things worked, where you can place bets. And I think that's something that I struggled to do earlier in my career where being very short term quarter focused, when all of a sudden the plan jumps 10%, 25%, 50% over the next couple of years, you need time to ramp up those channels.
Dave Gerhardt [00:33:33]:
You're not just going to be like, wow, we need 500 new hand raisers this quarter. Let's go try LinkedIn ads for the first time. It's not going to work. Back to the point about incrementality. You need to be able to do that stuff early on and test into those new channels. And that's where this is really important.
Pranav Piyush [00:33:51]:
I agree wholeheartedly. And something that we did not cover today is when you do this type of analysis, it helps you develop recommendations for your business that say things like, you probably are oversaturated in a certain channel and pouring more dollars into that channel is not going to get you incremental returns. So think about reallocating to this other channel because that one seems to be undersaturated things of that nature. Right? So you'll get that as an output from these type of analyses and whether you do it in house or you find a vendor to support you through it. And then the last piece, I will say about experimentation, to your point, I don't know if people are aware of this or not, but we looked at experimentation data from companies like booking.com and Amazon and Airbnb and Uber and Microsoft. 70% to 80% of experiments will fail. Let that sink in. Right? What that's saying is you designed ten experiments for this year, 70% to 80%.
Pranav Piyush [00:34:51]:
Seven to eight out of them are just going to fail. They're not going to bring in incremental returns. And that is something that's really hard to swallow if you are doing new stuff for the first time. But that is an expectation that you need to set with your marketing and finance leadership, that you need to have an experimentation budget. And the goal of that budget is not to get you slam dunk incremental returns. It's about finding the two or three things out of those ten that if they hit. They're massive.
Dave Gerhardt [00:35:19]:
All right, let's rip through these questions. Let me just pop in over here. All right, cool. This question is from Ian for b, two b ad spend. Do you believe that there is a shift to a media mix model rather than multi touch attribution?
Pranav Piyush [00:35:35]:
Absolutely. I mean, I'm obviously biased, right? This whole idea of using touches or clicks or tabs, to me, if you're not doing correlation or causation on top of that data set, then it's not really attribution in my opinion. So you want to call it attribution? You can. Lots of people do. But again, it goes back to, if there's no cause and effect relationship, then I personally wouldn't call that attribution. And yes, media mixed modeling is where.
Dave Gerhardt [00:36:02]:
The future is at. Great question here from Erica, related to what we just talked about. How do you think about short term incremental revenue versus long term investments where revenue is realized at a later date? Great question.
Pranav Piyush [00:36:14]:
This is the holy grail of marketing measurement. So I think of it in the following ways. Right. And some hot takes coming, so please don't judge. We put all of these different channels and tactics and marketing, we put prefixes on them, right? We call something brand marketing versus performance, marketing versus growth, marketing versus customer marketing. And some of that is because of this long term and short term, and people's inability to kind of see through the entire time horizon. And what I will tell you is, whether you call it long term or you call it short term, every piece of marketing is going to have some effect in the short term. Every piece of marketing.
Pranav Piyush [00:36:53]:
So this idea that some marketing only works in the long term is just completely not true. Every piece of marketing, if it is effective, is going to bring in returns in the short term. You just don't know how to measure it. And that's why you end up saying that, well, it's long term, and we're going to get that in three quarters. Nope, that's not how marketing works.
Dave Gerhardt [00:37:13]:
I want to build on that. This question is from Dave Gerhardt.
Pranav Piyush [00:37:16]:
Hi.
Dave Gerhardt [00:37:16]:
How do you think about measurement for non direct response initiatives? Kind of building on this, like it's easy to do a campaign or to run some marketing efforts on a specific channel. But I've been at companies where we've had channels like a podcast, where the goal of the podcast is not to drive customers, but because we had a podcast, we ended up getting a lot of customers from that. But maybe, you know where I'm going this, but it doesn't work like one to one. It's not like, oh, let's create more podcast episodes and we will get more customers, or let's do this type of thing on the podcast. I've always struggled with how do you put those in the mix? And we're even dealing with this now for exit five, right? Us writing on LinkedIn is the number one source of new members. However, if I only write about promotional posts to drive new members into exit five, it just doesn't work like that. And I'm curious to hear your take on that.
Pranav Piyush [00:38:11]:
Yeah, it's a great question. Right. So what you're really asking is there are things at the bottom of the funnel that have call to actions like try it now, start a free trial, buy now, book a demo. And what about all those marketing strategies that aren't bottom of funnel, that are top of funnel, that are awareness focused that we will call brand marketing awareness, whatever you want to call it, non direct response. All I'm saying is that good marketing, Dave, your podcasts or your LinkedIn posts are going to appeal to me to want to go into exit five anyway, even if you don't put a call to action and a direct response format to it. And this is the mental barrier that we have to get across. And so in terms of your measurement strategy, here's how it works. You look at impressions or reach across every single channel over your historical time period.
Pranav Piyush [00:39:09]:
How many people are actually downloading and listening to your podcast? How many people are actually reading your LinkedIn posts. That becomes an input into the marketing mix model that I talked about. And so you don't have to guess and wait for a direct response. Click on those pieces of content. Now I will add one more thing, which is this idea that you were just alluding to, that I will do more podcasts and therefore I will have more business. Well, of course not. Podcast itself is just an asset. The distribution and how many people are actually listening to the podcast is the input into the model.
Pranav Piyush [00:39:49]:
And that's where your unlock is going to be. If the same podcast is listened by ten people or 1000 people, obviously that's going to have an impact on your inbound lead volume.
Dave Gerhardt [00:39:59]:
Cool. And just building on that, I think being able to articulate where the podcast fits in your marketing strategy, it's like growing the number of people who might hear about us over time is going to increase in some number of exactly hand raisers. I used to always battle with our head of ops on this. He's like, well, how many demos do you think we're going to get from the podcast this year? I'm like, I don't know. None. We could get none. That's not the goal of it, but we happen to get some from it.
Pranav Piyush [00:40:24]:
Yeah, that's exactly right. Like, what is the goal of the thing that you're doing? And if it's not handraisers, well, what is it? Right, and let's talk about that. And early on my career, I struggled with this, just like you, Dave, because it was hard to say this is the goal, because I didn't have a clear way of saying I can measure the impact on that metric. What I'm telling you now is you absolutely can measure the impact on that. And I will also tell you, you're not going to get it right in one quarter. You got to give it time to experiment and figure out the distribution channels to get to the right ICP and hopefully have compelling content that then gets them inbound. So there's a multitude of factors, and we try to simplify it in a bite sized way, but you really have to be able to talk about this intelligently with your peer group as a marketer.
Dave Gerhardt [00:41:14]:
All right, I just saw this one. I want to build on this. So Christine asks, when do you know it's time to cut losses on experiments versus waiting long enough to see an impact?
Pranav Piyush [00:41:22]:
Oh, wow. Experiments. I'll tell you my way of thinking about it. And this is the way of thinking about experiments. You define your hypothesis, your budget, your time period, and your expected or desired effect on your metric before you do anything. Right? So let that sink in. Right? It's like, what's the hypothesis? How long are you going to run this test? How much budget are you allocating on the test? And what is the expected lift in the metric that you care about? When you set those four things up right off the bat, then it's very clear. At the end of the test period, you call it, did the experiment work or did it not work and what you learned from it? And you rinse and repeat.
Pranav Piyush [00:42:01]:
So it's not a subjective answer, it's an objective answer based on how you designed your experiment.
Dave Gerhardt [00:42:08]:
This question is from Zach. It would seem that how marketing is defined, as well as how a business is structured, is integral to how it's measured, particularly incrementality. Given that and the general nature of cross functional teams in BDB, do you see sales as a component of marketing, a partner to with marketing or its own function? And how does that influence how you measure success?
Pranav Piyush [00:42:29]:
Yeah, it's a great question. Sales is sales. Marketing is marketing. These are two distinct and different things, so no one is not part of the other. Whichever way you slice and dice it. Now, you might have different reporting structures because you have a unicorn of a business leader who can think equally about marketing and sales. Great. If you have that in your business and it makes sense for you to organize the team that way.
Pranav Piyush [00:42:55]:
I get it. I will also say that 90% of the cases, it's very hard to find that unicorn. So just keep that in mind. So I don't think about sales and marketing in this sort of unified way. Contrary to popular belief, what I would say is, go back to modeling your business realistically, and I'm going to jump back and kind of put that slide up here. Right. Because if you are understanding how your business model works and what marketing is responsible for, then all your sales and marketing alignment issues fade away. If you know that marketing is driving handraisers primarily, that is the key performance indicator of your marketing team, then, hey, why do you need a whole bunch more alignment between sales and marketing? And I'll just be very clear here.
Pranav Piyush [00:43:44]:
I hate the word qualified. Marketing qualified. Sales qualified. Your customers are qualifying you. You are nobody to qualify your customers like, sorry, that's a very strong sort of statement. But I strongly believe that if you throw that word out of your vocabulary and focus on the true, explicit intent of a prospect coming to your website, then you won't have the problem of figuring out how to get marketing and sales to work together. It's very clear. Got 20 people raising their hand and saying, give me a demo.
Dave Gerhardt [00:44:15]:
Cool. All right, do you have any parting thoughts before we get out of here? Give you the mic.
Pranav Piyush [00:44:19]:
Well, I appreciate that. We had a fun little thing going here for the last ten minutes. So folks who are still watching, we have a fun little game. So hopefully it'll work. We'll see. So Super bowl just went by in the US, so I apologize to everyone who's dialing in from outside of the US, but what do Super bowl ads do? How do you measure their impact? Right there are these really notorious but talked about things in advertising, so I thought it'd be know. I don't know if you all are Deadpool fans or not. I am.
Pranav Piyush [00:44:49]:
Love that movie and that character. And they had a pretty interesting ad, right? Wolverine and Deadpool. So I'm curious, if we had to take a poll in the chat, what do you think that ad did for Deadpool in terms of the effect it had on Deadpool? Any thoughts? Dave, what do you think huge impact.
Dave Gerhardt [00:45:07]:
Ticket sales, huge impact. Give us the.
Pranav Piyush [00:45:12]:
Basic spike and then nothing, basically. Right. So if you look at just Deadpool searches on Google trends before and after a Super bowl, there's a very minor impact. So if you think of this as a leading indicator. Right. Just people searching for Deadpool definitely had some impact. But if you don't follow that up with some sort of other ongoing campaign, you're going to realize that it didn't really do much. So that's Deadpool.
Pranav Piyush [00:45:38]:
I'm not saying don't advertise on the Super bowl, but it's just an interesting way of looking at the business. I thought this ad was hilarious. I don't know if you.
Dave Gerhardt [00:45:46]:
So I use this, thanks to my wife, I have a proper skincare routine because she's seen me on video and she's like, bro, we got to fix you up a little bit. So I got some Sarah V, among other things, and my daughter, who's six, we watched a little bit of Super bowl. She saw this commercial, she sees this on the counter. She said, hey, that's the lotion from the commercial. So there's at least one ROI measure right there. Okay.
Pranav Piyush [00:46:08]:
There you go. There you go. They actually did some really interesting stuff, micro influencer stuff before Super bowl and. Yeah, exactly. Natalie. The build up and the lead up was fantastic. So what did that do? It actually had, I would say a better impact than the Deadpool one, but also not that sustained. So you can kind of see, if you zoom into Google trends, you can go there and do that yourself.
Pranav Piyush [00:46:29]:
You'll see that the overall interest level did see spikes after the ad aired, but again, not a huge one. So there you go. State Farm. I don't know. I kind of like the ad and the throwback to the 90s with, I forget the name of the movie that these two appeared in, but any takers on what it did to State Farm? Twins. That's right. Yes. So not much.
Pranav Piyush [00:46:53]:
Nothing happened. State Farm has a lot of organic searches happening all the time. So this little spike here on a Saturday. So their weekends are pretty dull in terms of people going to State Farm or searching for State Farm. So you had a spike, but that spike was pretty much the same as all the other weekday spikes that they have. So that's what you get. So a couple more here. I thought by far the people who won Super bowl this year, apart from the chiefs, were Dunkin donuts.
Pranav Piyush [00:47:21]:
I thought their advertisement was awesome for people who don't know they've been doing this. Ben Affleck meme for, like, multiple years, and they're actually private equity owned, which means to get approval for a 20, $30 million advertising budget for Super bowl, you really have to know what you're doing and be able to prove the impact that it's going to have. Check out what it did. I just searched for dunkings, so not even Dunkin donuts. And you can see that they've created a universe. Know Ben Affleck and Tom Brady and Matt, and they can continue to leverage this over time. There was obviously no search mentions of dunkings, and now it's showing up, which is pretty awesome. All right, last one, Timu.
Pranav Piyush [00:48:03]:
So I was surprised by how many times this ad showed up on Super bowl. It was like they played it like, six or seven times. I don't know about you, but I thought, I was curious about what impact that would have, and the numbers won't do it justice, but there was a marked increase, a baseline level increase that you see before and after as a result of that. I don't know if that'll show up in sales and active users, but sometimes things that seem OD on the surface may actually have a pretty good impact if you know what you're doing. So check out Timo. Okay.
Dave Gerhardt [00:48:40]:
I still have no idea what this is. Is it a real website? I have no idea. No.
Pranav Piyush [00:48:44]:
You have no idea what Timo is? Oh, my God.
Dave Gerhardt [00:48:46]:
You got to shop like a billionaire. They also wrote a whole wave of just, like, social media memes based on this, which is also, I think, an opportunity.
Pranav Piyush [00:48:56]:
That's right. Okay. That's all we have. Connect with me on LinkedIn if you want to learn more about this type of stuff. I write about it, I talk about it, and then exit five. So plug here for the community. Go in there, and let's have a conversation and a group chat about how you all can leverage some of this in your day to day.
Dave Gerhardt [00:49:16]:
Awesome. Thanks, Pranav. Thank you, everybody, for hanging out here today. This was a great session. We got more of these. We try to do one or two of these a month. Go and connect with Pranav on LinkedIn. He is as responsive as he says, so hit him up if you have a follow up question.
Dave Gerhardt [00:49:29]:
And he's also an active member in the exit five community. So go in there, start a discussion about measurement. Otherwise, I'm out of here. I'll see you all have a great rest of the day. Great rest of the week. I'll see you all inside of the community and on LinkedIn and everywhere else. All right. See you later.
Pranav Piyush [00:49:42]:
See y'all close.