1 00:00:05,600 --> 00:00:11,460 Welcome to another edition of the Always Be Testing podcast with your host Tye De 2 00:00:11,460 --> 00:00:11,980 Grange. 3 00:00:12,400 --> 00:00:16,500 Get a guided tour of the world of growth, performance marketing, customer 4 00:00:16,500 --> 00:00:19,800 acquisition, paid media, and affiliate marketing. 5 00:00:20,440 --> 00:00:24,780 We talk with industry experts and discuss experiments and their learnings in 6 00:00:24,780 --> 00:00:27,000 growth, marketing, and life. 7 00:00:27,460 --> 00:00:32,000 Time to nerd out, check your biases at the door, and have some fun talking 8 00:00:32,000 --> 00:00:34,840 about data-driven growth and lessons learned. 9 00:00:36,540 --> 00:00:37,260 Hello. 10 00:00:37,260 --> 00:00:40,480 Welcome to the Always Be Testing podcast. 11 00:00:41,000 --> 00:00:45,820 I am absolutely thrilled to have Michael Kaminsky on as our guest. 12 00:00:46,580 --> 00:00:50,320 Always Be Testing podcast, where we talk about growth, performance marketing, 13 00:00:51,300 --> 00:00:56,360 testing, experimentation, learning, partner marketing, and all those things. 14 00:00:56,760 --> 00:00:58,960 And yeah, I'm thrilled to have Michael on today. 15 00:00:59,600 --> 00:01:05,740 Michael Kaminsky, the CEO of Recast, a modern approach to multimedia mixed 16 00:01:05,740 --> 00:01:08,440 modeling and all things attribution and incrementality. 17 00:01:08,920 --> 00:01:09,020 Welcome. 18 00:01:09,720 --> 00:01:10,440 Thanks for having me, Ty. 19 00:01:10,500 --> 00:01:13,840 I am very excited to be here, excited for our discussion today. 20 00:01:14,760 --> 00:01:15,720 Absolutely, absolutely. 21 00:01:15,940 --> 00:01:19,200 It's going to be a good one and I'm excited to dive in. 22 00:01:19,840 --> 00:01:21,900 This is, I feel like, such a hot button topic. 23 00:01:22,140 --> 00:01:26,780 For years, people have debated the topics of media mixed modeling, 24 00:01:27,140 --> 00:01:28,040 incrementality. 25 00:01:28,440 --> 00:01:30,060 It's a hot button topic. 26 00:01:30,240 --> 00:01:34,860 It's a topic that comes up for growth people, for data science nerds, for paid 27 00:01:34,860 --> 00:01:37,020 marketers, for affiliate and partner marketers. 28 00:01:37,800 --> 00:01:42,560 And you are the guy to talk to because you've launched a business that solely 29 00:01:42,560 --> 00:01:43,460 focuses on this. 30 00:01:43,620 --> 00:01:45,340 So let's just jump in. 31 00:01:45,440 --> 00:01:47,200 What the heck is media mixed modeling? 32 00:01:48,000 --> 00:01:49,540 Why is it so hot right now? 33 00:01:50,180 --> 00:01:51,060 What's going on? 34 00:01:51,660 --> 00:01:52,700 Yeah, great question. 35 00:01:53,020 --> 00:01:57,420 So I think everyone who's listening to this podcast can be familiar with a 36 00:01:57,420 --> 00:02:00,020 bunch of the changes that have been happening in the industry over the last 37 00:02:00,020 --> 00:02:04,580 couple of years, largely related to reduced ability to track people across the 38 00:02:04,580 --> 00:02:08,460 internet because of changes with app tracking transparency from Apple that 39 00:02:08,460 --> 00:02:12,540 rolled out with iOS 14.5, more changes to the ability to track that are coming 40 00:02:12,540 --> 00:02:19,320 with iOS 17, plus increased use of ad blockers, privacy regulations like GDPR. 41 00:02:19,600 --> 00:02:24,060 It has made the ability to track people across the internet consistently much, 42 00:02:24,200 --> 00:02:25,220 much, much harder. 43 00:02:25,800 --> 00:02:28,660 And so that means that a lot of the tools that a lot of marketers, and 44 00:02:28,660 --> 00:02:32,860 especially digital-focused marketers, got used to using over the last five or 45 00:02:32,860 --> 00:02:37,120 10 years, all of the digital tracking tools, first-touch attribution, last 46 00:02:37,120 --> 00:02:40,700 -touch attribution, all of those sorts of tools, they're now less reliable. 47 00:02:41,260 --> 00:02:45,540 And marketers know that, and they're starting to see the flaws in those 48 00:02:45,540 --> 00:02:50,020 methodologies that have really been made much more acute in the last couple of 49 00:02:50,020 --> 00:02:52,080 years with the changes to tracking. 50 00:02:52,400 --> 00:02:56,040 And so marketers have started to get really interested in, what are the 51 00:02:56,040 --> 00:02:59,760 alternative ways of measuring marketing performance that doesn't just rely on 52 00:02:59,760 --> 00:03:00,480 digital tracking? 53 00:03:01,060 --> 00:03:05,560 And marketing mix modeling is one of those forms of measuring marketing 54 00:03:05,560 --> 00:03:06,080 performance. 55 00:03:06,380 --> 00:03:10,680 It's basically an econometric model that looks at patterns in the data to try 56 00:03:10,680 --> 00:03:14,240 to understand the true incrementality of marketing performance. 57 00:03:15,100 --> 00:03:19,300 And because marketers no longer feel that they can just rely on digital 58 00:03:19,300 --> 00:03:22,840 tracking, it's becoming a thing that a lot of people are talking about. 59 00:03:23,080 --> 00:03:28,440 That being said, because marketing mix modeling, a lot of the old ways of doing 60 00:03:28,440 --> 00:03:32,800 it were not really built for modern digital marketing, there's been a lot of 61 00:03:32,800 --> 00:03:35,280 technologies that have been developed in the last couple of years of people 62 00:03:35,280 --> 00:03:39,080 trying to figure out, how do we make marketing mix modeling actually work for 63 00:03:39,080 --> 00:03:43,140 the problems that marketers, modern marketers face today? 64 00:03:43,500 --> 00:03:44,620 And that's what we're doing at Recast. 65 00:03:44,800 --> 00:03:47,980 And that's why everyone is sort of talking about this and trying to figure out 66 00:03:47,980 --> 00:03:51,920 how do we actually make this tool work for us with all of the problems that 67 00:03:51,920 --> 00:03:54,660 we're facing today in 2023 as marketers. 68 00:03:54,960 --> 00:03:55,840 That's awesome. 69 00:03:56,240 --> 00:04:02,020 Yeah, it certainly seems to be insane demand for the right type of brand, the 70 00:04:02,020 --> 00:04:03,400 right type of situation. 71 00:04:03,900 --> 00:04:06,240 Just thinking back, when did this all start? 72 00:04:06,380 --> 00:04:11,780 The notion of media mix modeling has been around for quite a long time. 73 00:04:12,340 --> 00:04:13,260 Yeah, that's exactly right. 74 00:04:13,420 --> 00:04:15,519 So it's a really interesting history. 75 00:04:16,060 --> 00:04:19,060 So media mix modeling, marketing mix modeling has been around almost as long as 76 00:04:19,060 --> 00:04:20,160 computers have been around. 77 00:04:20,660 --> 00:04:26,240 If you think back to pre-e-commerce, pre-internet days, and you're the CMO of a 78 00:04:26,240 --> 00:04:32,200 CPG company like Pepsi or Gillette or whatever, every year you need to make 79 00:04:32,200 --> 00:04:35,760 decisions about how you're going to allocate your marketing budget across the 80 00:04:35,760 --> 00:04:37,460 different ways that you can spend it. 81 00:04:37,540 --> 00:04:42,040 And that might be with doing promotional activity or running TV ads or running 82 00:04:42,040 --> 00:04:43,860 print ads or running radio ads. 83 00:04:44,100 --> 00:04:46,020 And there's no digital tracking at all. 84 00:04:46,120 --> 00:04:47,060 People don't buy online. 85 00:04:47,180 --> 00:04:50,560 The internet doesn't really exist, at least not the way that it exists today. 86 00:04:50,700 --> 00:04:52,520 And so how are you going to make that decision? 87 00:04:52,720 --> 00:04:57,360 And the way that CMOs made that decision historically, especially in fast 88 00:04:57,360 --> 00:05:01,520 moving consumer goods industries, was through these econometric projects. 89 00:05:01,640 --> 00:05:04,220 Effectively, you would hire a statistician or an econometrician. 90 00:05:04,300 --> 00:05:05,600 They would look at your historical data. 91 00:05:06,080 --> 00:05:11,000 At times when you spent more on TV, how many additional Pepsis or Razors did 92 00:05:11,000 --> 00:05:11,420 you sell? 93 00:05:11,620 --> 00:05:16,760 At times when you spend more on radio, how many additional Pepsis or Razors do 94 00:05:16,760 --> 00:05:17,200 you sell? 95 00:05:17,660 --> 00:05:21,920 And so these econometricians would do these research projects once a year. 96 00:05:22,000 --> 00:05:22,760 They were very expensive. 97 00:05:23,420 --> 00:05:25,500 These were highly trained PhDs. 98 00:05:25,560 --> 00:05:27,060 They were based on consulting projects. 99 00:05:27,220 --> 00:05:27,820 They would come in. 100 00:05:27,900 --> 00:05:28,680 They would do this analysis. 101 00:05:29,280 --> 00:05:31,700 Lots and lots of research and investigation goes into it. 102 00:05:32,040 --> 00:05:32,840 They produce a report. 103 00:05:33,340 --> 00:05:34,960 And then the CMO looks at it and says, OK, great. 104 00:05:34,960 --> 00:05:39,100 We're going to allocate $10 million to TV and $15 million to radio and $5 105 00:05:39,100 --> 00:05:39,940 million to print. 106 00:05:40,340 --> 00:05:44,520 And then the marketers would go out and buy that media at upfronts once a year. 107 00:05:44,680 --> 00:05:48,140 And then the marketing team would go out and sort of execute on that plan with 108 00:05:48,140 --> 00:05:49,300 all of the media that they had bought. 109 00:05:49,580 --> 00:05:53,780 And so that's where this idea came from, is how do we measure marketing 110 00:05:53,780 --> 00:05:55,840 effectiveness when we don't track anyone at all? 111 00:05:55,920 --> 00:05:58,400 We don't have any digital tracking to rely on. 112 00:05:58,980 --> 00:06:05,660 And so MMM, media mixed modeling, has been around for 50 or 60 years as these 113 00:06:05,660 --> 00:06:09,180 types of fast-moving consumer good brands have needed a way to measure 114 00:06:09,180 --> 00:06:12,200 marketing effectiveness without being able to digitally track someone. 115 00:06:12,740 --> 00:06:15,480 Over the last 10 or 15 years, it sort of fell out of favor a little bit, 116 00:06:15,580 --> 00:06:18,700 especially with the rise of e-commerce, where we could track people. 117 00:06:19,060 --> 00:06:23,140 But now that our ability to track is again reduced, it's coming back into 118 00:06:23,140 --> 00:06:23,460 favor. 119 00:06:24,020 --> 00:06:29,620 But we all recognize that the old way of doing it, where you hire a bunch of 120 00:06:29,620 --> 00:06:32,860 statisticians and they produce a report once every six months or once a year, 121 00:06:33,000 --> 00:06:36,480 doesn't really match the way that modern marketing actually works. 122 00:06:37,000 --> 00:06:37,440 That's amazing. 123 00:06:37,580 --> 00:06:40,700 Yeah, it's almost come full circle, it sounds like, right? 124 00:06:41,100 --> 00:06:41,700 Yeah, totally. 125 00:06:41,840 --> 00:06:44,680 Well, I mean, what's old is new again, right? 126 00:06:44,720 --> 00:06:46,420 And this is the way that marketing gets to work. 127 00:06:46,780 --> 00:06:50,500 But I think the key thing here is that we have to figure out what are we 128 00:06:50,500 --> 00:06:52,600 actually trying to do with this technology? 129 00:06:53,060 --> 00:06:56,620 And then how can we get it into the hands of more marketers so they can 130 00:06:56,620 --> 00:06:58,040 actually use it to make real decisions? 131 00:06:58,240 --> 00:07:01,980 That's what the core thing is that we're trying to do here, is how do we 132 00:07:01,980 --> 00:07:06,340 actually drive businesses forward by helping them accurately measure their 133 00:07:06,340 --> 00:07:07,160 marketing effectiveness? 134 00:07:07,360 --> 00:07:08,720 That's the most important thing. 135 00:07:08,920 --> 00:07:12,540 And when we're thinking about MarTech tools, any sort of attribution 136 00:07:12,540 --> 00:07:16,140 methodology, we always want to be thinking about how are we using this to drive 137 00:07:16,140 --> 00:07:17,160 the business forward? 138 00:07:17,520 --> 00:07:20,700 How are we using it to actually estimate incrementality so that we can spend 139 00:07:20,700 --> 00:07:22,120 our dollars where they're most effective? 140 00:07:22,500 --> 00:07:27,760 And is incrementality like the kind of big value that really is derived from 141 00:07:27,760 --> 00:07:28,080 MMM? 142 00:07:28,600 --> 00:07:33,500 So in my view, incrementality should be really the only thing that matters for 143 00:07:33,500 --> 00:07:33,920 marketers. 144 00:07:34,440 --> 00:07:38,940 Marketers should be focused on how are we measuring incrementality for our 145 00:07:38,940 --> 00:07:39,340 business? 146 00:07:39,800 --> 00:07:42,100 And let's maybe take a step back for the listeners. 147 00:07:42,360 --> 00:07:44,140 What do we mean when we say incrementality? 148 00:07:44,560 --> 00:07:48,720 For me, incrementality means if I spend an additional dollar in this marketing 149 00:07:48,720 --> 00:07:52,500 channel, how much additional revenue is that dollar going to drive? 150 00:07:52,760 --> 00:07:56,160 Or if I pull a dollar out of some marketing channel, how much revenue are we 151 00:07:56,160 --> 00:07:59,640 going to lose from having pulled that dollar out of that channel? 152 00:08:00,040 --> 00:08:01,440 That's what incrementality means to us. 153 00:08:01,460 --> 00:08:05,380 It is the causal relationship between the marketing activity that we're doing 154 00:08:05,380 --> 00:08:07,680 and then the business results on the other side. 155 00:08:07,800 --> 00:08:09,380 And that's the thing that we want to understand. 156 00:08:09,760 --> 00:08:13,360 Because if we understand that, if we understand true incrementality, then we 157 00:08:13,360 --> 00:08:15,600 can actually optimize our marketing budget. 158 00:08:15,600 --> 00:08:19,240 Because we can take our whole marketing budget and then allocate the dollars to 159 00:08:19,240 --> 00:08:20,480 where they're all most effective. 160 00:08:20,760 --> 00:08:22,960 And that's the best that we can possibly do as marketers. 161 00:08:23,380 --> 00:08:25,740 Of course, there's a bunch of complications that go around that. 162 00:08:26,020 --> 00:08:30,060 But in the ideal scenario, that's what we're aiming for when we're talking 163 00:08:30,060 --> 00:08:31,440 about doing marketing measurement. 164 00:08:32,039 --> 00:08:33,980 And so incrementality is the thing. 165 00:08:34,340 --> 00:08:36,940 Everyone should always only be thinking about incrementality. 166 00:08:36,980 --> 00:08:40,100 In all of these different measurement methods, we should be thinking about is 167 00:08:40,100 --> 00:08:42,480 this getting us closer to incrementality or not? 168 00:08:42,960 --> 00:08:46,660 Or this measurement method works in this case, but in this other case, it's not 169 00:08:46,660 --> 00:08:48,180 actually good at measuring incrementality. 170 00:08:48,660 --> 00:08:52,940 And so that's what I want marketers to be thinking about is how is this 171 00:08:52,940 --> 00:08:56,180 measuring incrementality and how is it getting us closer to that ideal state of 172 00:08:56,180 --> 00:08:59,860 being able to really understand the true causal relationships between our 173 00:08:59,860 --> 00:09:02,100 marketing activity and our business performance. 174 00:09:03,400 --> 00:09:06,080 So incrementality, hugely important concept. 175 00:09:06,620 --> 00:09:09,900 MMM is one way of measuring incrementality. 176 00:09:10,120 --> 00:09:13,520 And if you do MMM right and you're thinking about it the right way, you should 177 00:09:13,520 --> 00:09:17,660 be trying to use MMM to measure incrementality the same way that you should be 178 00:09:17,660 --> 00:09:20,380 using experimentation to try to measure incrementality. 179 00:09:20,740 --> 00:09:23,380 The same way that you should be thinking about how does our last touch 180 00:09:23,380 --> 00:09:25,900 attribution not measure incrementality? 181 00:09:26,040 --> 00:09:30,080 And in what cases should we not be relying on this tracking methodology to 182 00:09:30,080 --> 00:09:32,440 actually understand the true incrementality of what we're doing? 183 00:09:32,800 --> 00:09:36,140 All of those are the sorts of questions that I think that marketers should 184 00:09:36,140 --> 00:09:37,060 really be focused on. 185 00:09:37,440 --> 00:09:38,300 Yeah, I love that. 186 00:09:38,380 --> 00:09:42,840 And I love the concept of like, hey, if this campaign marketing lever were to 187 00:09:42,840 --> 00:09:47,580 be removed, would you have received that conversion event or value or revenue 188 00:09:47,580 --> 00:09:48,800 without it? 189 00:09:48,800 --> 00:09:49,240 Exactly. 190 00:09:49,380 --> 00:09:52,920 It's a good kind of framework as well to build on your definition of 191 00:09:52,920 --> 00:09:53,600 incrementality. 192 00:09:53,880 --> 00:09:56,900 And honestly, I'm really excited to talk with you, Tai, because I know that 193 00:09:56,900 --> 00:10:00,580 you're an expert in like the affiliate marketing space. 194 00:10:00,660 --> 00:10:05,360 And the affiliate marketing space is one, I think like a lot of people don't 195 00:10:05,360 --> 00:10:10,080 really understand the full range of affiliate marketing and what's possible 196 00:10:10,080 --> 00:10:11,900 with affiliate marketing. 197 00:10:12,300 --> 00:10:16,960 But also, it's sort of famously a really difficult channel to measure the 198 00:10:16,960 --> 00:10:17,660 incrementality of. 199 00:10:17,740 --> 00:10:20,240 This is a thing that we've spent a lot of time thinking about at Recast. 200 00:10:20,360 --> 00:10:22,540 I know you have spent a lot of time thinking about it. 201 00:10:22,560 --> 00:10:25,280 So I'm really excited to have this conversation today because I want to ask you 202 00:10:25,280 --> 00:10:30,780 about what are the best practices around measuring the true incrementality of 203 00:10:30,780 --> 00:10:31,840 these affiliate programs? 204 00:10:32,140 --> 00:10:35,960 How can marketers get smart about that in a world where it just it feels like 205 00:10:35,960 --> 00:10:40,340 it's a lot more difficult to test for a bunch of different reasons, a bunch of 206 00:10:40,340 --> 00:10:42,080 tricky complications that go into it. 207 00:10:42,480 --> 00:10:44,680 So maybe I guess like, I'd love to hear from you, Tai. 208 00:10:45,320 --> 00:10:49,600 What are the main things that people don't understand about affiliate marketing 209 00:10:49,600 --> 00:10:53,580 and maybe talk a little bit about like, why is it so hard to test these 210 00:10:53,580 --> 00:10:54,460 affiliate channels? 211 00:10:55,020 --> 00:10:56,280 I see what you did there, Mike. 212 00:10:56,340 --> 00:10:58,580 You're flipping it on the interviewer interviewing me. 213 00:10:58,680 --> 00:10:59,420 I get it. 214 00:10:59,620 --> 00:10:59,880 Okay. 215 00:11:00,140 --> 00:11:01,940 I want to get something out of this conversation too. 216 00:11:02,000 --> 00:11:02,660 This is for me. 217 00:11:02,860 --> 00:11:03,300 Absolutely. 218 00:11:03,920 --> 00:11:08,160 Yeah, it's so funny because I feel like affiliate and I've been sharing this a 219 00:11:08,160 --> 00:11:13,080 lot is the most misunderstood performance marketing lever and likely the most 220 00:11:13,080 --> 00:11:14,680 underrated as a result of that. 221 00:11:14,900 --> 00:11:20,300 A lot of the misconceptions are also stem around the topic of incrementality 222 00:11:20,300 --> 00:11:22,060 and other related topics. 223 00:11:22,900 --> 00:11:28,840 Historically, brands have tried to sort through the challenges of fraud. 224 00:11:29,000 --> 00:11:32,160 I think that's common for any performance marketing channel and digital 225 00:11:32,160 --> 00:11:35,840 channels, especially in their early days as they matured, that fraud typically 226 00:11:35,840 --> 00:11:38,360 got better tools caught up. 227 00:11:38,660 --> 00:11:41,520 People got caught up in how to catch it and improve it. 228 00:11:42,020 --> 00:11:47,620 You had misaligned network incentives from the 2000s and 2010s where they were 229 00:11:47,620 --> 00:11:48,740 charging high rates. 230 00:11:48,840 --> 00:11:50,260 They could command higher rates. 231 00:11:50,620 --> 00:11:55,480 Sometimes they were aligned with paying out partners more rather than paying 232 00:11:55,480 --> 00:12:00,660 out what was efficient and accurate and rewarded and valued in terms of what 233 00:12:00,660 --> 00:12:01,280 was tracked. 234 00:12:02,060 --> 00:12:06,360 There's a lot of things that surround the affiliate space that get 235 00:12:06,360 --> 00:12:10,260 misconstrued, misunderstood, that are historical, that are real, that are 236 00:12:10,260 --> 00:12:12,300 current, that are perception versus reality. 237 00:12:12,880 --> 00:12:14,880 You've got coupon and deal sites. 238 00:12:15,080 --> 00:12:18,060 The theory and some of the perception is that that's what they are. 239 00:12:18,140 --> 00:12:19,280 That's what affiliate is. 240 00:12:19,920 --> 00:12:22,340 They're simply taking credit at the last minute. 241 00:12:22,800 --> 00:12:27,000 There's some truth to that, but that's certainly not what affiliate all is. 242 00:12:27,380 --> 00:12:30,200 When I think about affiliate, I definitely think about these coupon sites. 243 00:12:30,480 --> 00:12:33,840 Whenever I'm making a purchase online, when I'm at the checkout, I always 244 00:12:33,840 --> 00:12:36,460 Google coupon and I go to a coupon site and I grab the coupon. 245 00:12:36,720 --> 00:12:41,340 I totally see that case as being like, look, there's a good chance this isn't 246 00:12:41,340 --> 00:12:44,700 truly incremental because I'm going to buy whether I get the coupon or not, but 247 00:12:44,700 --> 00:12:46,180 if I get the extra 15%, great. 248 00:12:46,800 --> 00:12:51,140 What are the other types of affiliate that people maybe aren't thinking about? 249 00:12:51,380 --> 00:12:55,600 I mean, the reality is under partner marketing, under affiliate marketing, 250 00:12:55,600 --> 00:12:58,380 under influencer marketing, I consider them all the same. 251 00:12:59,040 --> 00:13:02,760 Fundamentally, when it comes to the work and the scope and the specifics of the 252 00:13:02,760 --> 00:13:07,480 actions we do for our clients, it's affiliate and influencer are different, but 253 00:13:07,480 --> 00:13:09,020 fundamentally they're the same. 254 00:13:09,340 --> 00:13:13,140 When you look at affiliate marketing, it's a way for you to reach people multi 255 00:13:13,140 --> 00:13:18,760 -channel across multiple touch points of Google, Meta, TikTok, review sites, 256 00:13:19,000 --> 00:13:20,660 gift guides, media houses. 257 00:13:21,360 --> 00:13:26,260 Think apartment therapy, BuzzFeed, Sports Illustrated, American Express, 258 00:13:26,420 --> 00:13:26,840 rewards. 259 00:13:27,580 --> 00:13:32,380 There's just thousands of quality content touch points to reach consumers via 260 00:13:32,380 --> 00:13:36,320 affiliate and affiliate in its definition really is saying, hey, I want to pay 261 00:13:36,320 --> 00:13:37,020 for outcomes. 262 00:13:37,160 --> 00:13:40,360 I want to pay for valued outcomes, not just clicks and eyeballs. 263 00:13:40,740 --> 00:13:44,340 You think about Meta and Google, essentially that's the payment mechanism that 264 00:13:44,340 --> 00:13:45,380 you're opting into. 265 00:13:46,020 --> 00:13:52,800 Yes, you're backing out to a valued ROI, ROAS, MER, CPA, whatever that KPI 266 00:13:52,800 --> 00:13:53,920 might be for your business. 267 00:13:54,280 --> 00:13:58,800 But with affiliate, you're able to say for a much larger percentage of the 268 00:13:58,800 --> 00:14:03,260 action and a much larger percentage of the budget, I can pay for an outcome and 269 00:14:03,260 --> 00:14:05,220 say, I need to pay a $20 CPA. 270 00:14:05,620 --> 00:14:08,820 I need to pay $5 a lead, whatever that might be for the brand. 271 00:14:09,260 --> 00:14:09,420 Totally. 272 00:14:09,580 --> 00:14:09,760 Okay. 273 00:14:10,160 --> 00:14:10,940 That makes sense. 274 00:14:11,060 --> 00:14:11,880 Paying for outcomes. 275 00:14:12,020 --> 00:14:15,660 I think it's easier for me to stick case that say like, look, an affiliate deal 276 00:14:15,660 --> 00:14:22,180 with BuzzFeed could be hugely valuable for a brand when the honey toolbar thing 277 00:14:22,180 --> 00:14:23,140 is maybe not. 278 00:14:23,420 --> 00:14:25,140 But how do we think about measuring that? 279 00:14:25,220 --> 00:14:27,380 How do we prove that as a marketer? 280 00:14:27,600 --> 00:14:30,620 If I'm an affiliate marketer, how do I go prove that to my boss? 281 00:14:31,180 --> 00:14:34,720 I think this is a thing that I've spent a bunch of time thinking about. 282 00:14:34,760 --> 00:14:37,360 It just feels very hard to test this sort of channel. 283 00:14:37,740 --> 00:14:43,800 With a Facebook, we can show ads to some people and not to others, or we can 284 00:14:43,800 --> 00:14:47,820 run Facebook ads in Washington, but not California, and you can start to think 285 00:14:47,820 --> 00:14:48,580 about running experiments. 286 00:14:48,580 --> 00:14:51,940 But with affiliate, that seems a lot harder. 287 00:14:52,060 --> 00:14:56,860 How do you think about actually doing that to prove the value of this marketing 288 00:14:56,860 --> 00:14:57,340 activity? 289 00:14:57,580 --> 00:15:01,420 Yeah, you don't really have the closed loop ecosystem of meta or Google. 290 00:15:01,680 --> 00:15:06,460 You do have a network, like an impact that'll track all your partner activity 291 00:15:06,460 --> 00:15:07,280 for the most part. 292 00:15:08,180 --> 00:15:12,800 That's a win and that's an opportunity to kind of look into the data and really 293 00:15:12,800 --> 00:15:13,480 dive into that. 294 00:15:13,600 --> 00:15:14,120 But you're right. 295 00:15:14,360 --> 00:15:16,300 There are challenges with it. 296 00:15:16,840 --> 00:15:19,860 It's not as impossible as some might think, though. 297 00:15:20,620 --> 00:15:22,980 I think that's a big call out. 298 00:15:23,660 --> 00:15:24,680 Don't lose hope. 299 00:15:25,020 --> 00:15:28,000 We'll kind of dive in and work through it together here and figure it out. 300 00:15:28,220 --> 00:15:31,880 But the reasons why it's hard are long and numerous. 301 00:15:32,340 --> 00:15:34,080 It's not an easy thing. 302 00:15:34,200 --> 00:15:37,480 I think the toolbars you referenced are a big part of that. 303 00:15:37,740 --> 00:15:41,320 Historically, that's been debated for probably about 20 years. 304 00:15:42,140 --> 00:15:43,880 There needs to be, I think, more... 305 00:15:43,880 --> 00:15:47,300 In a matter of fact, just at a macro level, the performance marketing agency 306 00:15:47,300 --> 00:15:52,400 I'm part of is running studies and really exploring a lot of this and 307 00:15:52,400 --> 00:15:57,520 resurfacing the topic of toolbars like Honey to say, okay, what type of data do 308 00:15:57,520 --> 00:16:03,600 people have access to transparently to see how much traffic came from a toolbar 309 00:16:03,600 --> 00:16:08,900 -like partner or browser plugin versus maybe other traffic sources that partner 310 00:16:08,900 --> 00:16:09,480 can provide. 311 00:16:09,580 --> 00:16:12,400 Ultimately, we are very cautious. 312 00:16:12,960 --> 00:16:14,240 We are very wary. 313 00:16:14,960 --> 00:16:17,480 We take a very judicious approach. 314 00:16:17,620 --> 00:16:21,940 If we work with a partner like that, sometimes we inherit a partner like that 315 00:16:21,940 --> 00:16:23,920 when a client has it already in their program. 316 00:16:24,580 --> 00:16:29,480 And we want to look at a lot of different data sources to determine if this 317 00:16:29,480 --> 00:16:31,540 makes sense for their overall strategy. 318 00:16:31,680 --> 00:16:34,220 A lot of things are kind of required to think about in there. 319 00:16:34,820 --> 00:16:36,200 What stage are they in? 320 00:16:36,320 --> 00:16:38,180 Are they in a conquesting stage? 321 00:16:38,400 --> 00:16:40,100 Are they in an aggressive spend stage? 322 00:16:40,660 --> 00:16:44,620 Are they trying to be very careful about profitability and maybe they've 323 00:16:44,620 --> 00:16:47,100 reached a maturity level or level of brand awareness? 324 00:16:47,660 --> 00:16:51,600 Those are important macro factors that play into if you would want to leverage 325 00:16:51,600 --> 00:16:52,540 a toolbar at all. 326 00:16:53,060 --> 00:16:58,740 For us, it makes it even more complicated when a lot of brands are not yet 327 00:16:58,740 --> 00:17:02,060 adopting anything beyond a last-click attribution model. 328 00:17:02,740 --> 00:17:07,319 That really hamstrings the efforts to really accurately... 329 00:17:07,319 --> 00:17:10,140 Obviously, this is different than incrementality, but it's related. 330 00:17:10,720 --> 00:17:15,500 You have this vast improvement in content, this vast improvement of quality, 331 00:17:16,000 --> 00:17:20,099 yet you're not attributing value to more than just the last click. 332 00:17:20,560 --> 00:17:23,700 What a missed opportunity for brands in the affiliate marketing space. 333 00:17:24,180 --> 00:17:27,180 And so I think that's a really important piece of this. 334 00:17:27,520 --> 00:17:32,300 That's next step piece that people need to get to in terms of figuring out and 335 00:17:32,300 --> 00:17:36,540 making sense of this before they can really proceed with, I think, the 336 00:17:36,540 --> 00:17:37,660 incrementality question. 337 00:17:38,160 --> 00:17:43,300 Just having the baseline stuff right is surprisingly not there. 338 00:17:43,580 --> 00:17:48,540 Not surprising to me, given how I've seen the data and marketing measurement 339 00:17:48,540 --> 00:17:49,680 setups of a lot of brands. 340 00:17:50,400 --> 00:17:54,240 I'm curious, tell me about some of these studies that you have run with these 341 00:17:54,240 --> 00:17:55,060 different affiliates. 342 00:17:55,220 --> 00:17:56,860 Maybe it's like, hey... 343 00:17:56,860 --> 00:18:00,100 I don't know what you've run, so I'm really curious to hear some examples of 344 00:18:00,100 --> 00:18:00,700 this in the wild. 345 00:18:00,940 --> 00:18:03,620 But have you worked with a brand that's using a toolbar and then you're like, 346 00:18:03,680 --> 00:18:06,280 let's turn off the toolbar for a month and see what happens? 347 00:18:06,800 --> 00:18:09,800 What are the different flavors of that that you run and what are the results 348 00:18:09,800 --> 00:18:10,380 that you've seen? 349 00:18:10,780 --> 00:18:15,780 Yeah, we've seen a number, both from an in-house as a marketer, in-house at 350 00:18:15,780 --> 00:18:20,700 large companies, as an agency, and talking to folks closely in the industry. 351 00:18:20,940 --> 00:18:22,060 We've seen a number of things. 352 00:18:22,060 --> 00:18:27,420 And I think the challenge is, oftentimes, it's hard to get a true read because 353 00:18:27,420 --> 00:18:33,440 running a true holdout is often, as you know, one of the best ways when you're 354 00:18:33,440 --> 00:18:37,020 not doing MMM to be able to measure for incrementality. 355 00:18:37,580 --> 00:18:42,140 And so as a result, when you don't control the environment like you do on Meta 356 00:18:42,140 --> 00:18:46,780 and Google, that really becomes harder geographically from a time perspective. 357 00:18:46,780 --> 00:18:49,080 Some brands will do kind of like a before and after. 358 00:18:49,900 --> 00:18:52,260 And as you know, that's inherently flawed. 359 00:18:53,160 --> 00:18:57,480 So I think what often happens is people get false positives or false negatives 360 00:18:57,480 --> 00:19:02,860 because they're not really understanding the science around incrementality and 361 00:19:02,860 --> 00:19:06,420 how you need to really look at that test in a very clean way. 362 00:19:07,000 --> 00:19:11,540 And so I have seen a movement towards more and more willingness and 363 00:19:11,540 --> 00:19:13,660 collaboration to run holdout tests. 364 00:19:14,160 --> 00:19:18,840 The challenge comes in the fact that a lot of partners, it requires partner 365 00:19:18,840 --> 00:19:23,560 participation, partner collaboration, the partner that is essentially at risk 366 00:19:23,560 --> 00:19:27,180 of losing out on a partnership, typically with a larger brand, because those 367 00:19:27,180 --> 00:19:30,420 are the brands that have the incentive, time and effort, money and budget to 368 00:19:30,420 --> 00:19:31,960 actually run a test like this. 369 00:19:32,720 --> 00:19:34,720 So it's really interesting. 370 00:19:35,320 --> 00:19:36,780 I do think it does require... 371 00:19:37,660 --> 00:19:41,500 So what often happens in the lead up to becoming that big brand is often we 372 00:19:41,500 --> 00:19:46,500 apply workarounds as best we can, like really effective pricing, looking at 373 00:19:46,500 --> 00:19:51,920 cohort analysis over time to see, hey, what quality type, what new customer, 374 00:19:52,020 --> 00:19:55,860 new to file customer data came through to a particular brand. 375 00:19:56,120 --> 00:19:59,600 Maybe we do try to turn off strategy and kind of see. 376 00:19:59,740 --> 00:20:02,600 And there's a surprising number of brands that actually will say, let's turn it 377 00:20:02,600 --> 00:20:05,460 back on based on their internal data that they'll see. 378 00:20:06,020 --> 00:20:11,780 So it's not necessarily just all full hearty people chasing dumb money. 379 00:20:12,140 --> 00:20:16,100 There is a level of brands that are saying, hey, we do want to opt back into 380 00:20:16,100 --> 00:20:19,700 this, whether that's maybe it's conquesting vis-a-vis customers. 381 00:20:19,900 --> 00:20:27,140 But I think to get to true MMM level, true data science level incrementality, I 382 00:20:27,140 --> 00:20:29,800 think we have to be smarter about running more holdout tests. 383 00:20:29,960 --> 00:20:32,060 I think the industry still has a ways to go. 384 00:20:32,700 --> 00:20:35,980 But I do believe that that's been the most effective I've seen. 385 00:20:36,380 --> 00:20:42,560 Let's say that I'm a brand and I've got a normal mix right now. 386 00:20:42,660 --> 00:20:44,500 So it's a lot of Facebook, a lot of digital marketing. 387 00:20:44,760 --> 00:20:46,240 Maybe I've got some offline stuff. 388 00:20:47,400 --> 00:20:51,420 And I'm interested, like, okay, affiliate might be a new growth path for us. 389 00:20:51,720 --> 00:20:56,580 What would you recommend that I do as a marketer at this brand to go test 390 00:20:56,580 --> 00:20:59,060 affiliate and figure out if it's going to work for us? 391 00:20:59,060 --> 00:20:59,880 Totally. 392 00:21:00,140 --> 00:21:05,800 I have a checklist of things, both published and in my brain with our team that 393 00:21:05,800 --> 00:21:06,780 we go through. 394 00:21:07,840 --> 00:21:12,580 I think I can run through that real quickly and then jump into what's there. 395 00:21:12,680 --> 00:21:17,400 But I think the obvious product market fit, obvious level of revenue needs to 396 00:21:17,400 --> 00:21:17,860 be there. 397 00:21:18,040 --> 00:21:23,680 Typically, a level of retention in the client, that flat retention curve that 398 00:21:23,680 --> 00:21:27,220 we talk about in growth, that folks are coming back, there's value there. 399 00:21:27,380 --> 00:21:31,240 There's not crazy high return levels or issues with that. 400 00:21:31,700 --> 00:21:34,400 The healthy conversion is important in affiliate because if you think about 401 00:21:34,400 --> 00:21:38,420 affiliate, they are putting skin in the game, time, money, expertise to promote 402 00:21:38,420 --> 00:21:39,040 your brand. 403 00:21:39,540 --> 00:21:42,580 And then expecting that return in the form of the commission as opposed to, 404 00:21:42,720 --> 00:21:44,180 hey, I'm getting money up front. 405 00:21:44,260 --> 00:21:47,640 Yes, both happen, but more often on the commission side in the affiliate space. 406 00:21:47,740 --> 00:21:49,720 So as a result, that conversion rate needs to be healthy. 407 00:21:49,720 --> 00:21:54,100 You go through that checklist top to bottom and say, hey, let's go forth and 408 00:21:54,100 --> 00:21:54,480 conquer. 409 00:21:55,060 --> 00:21:56,120 Time is important too. 410 00:21:56,280 --> 00:22:00,120 We need to be able to give this a real, it's relationship-based, it's some call 411 00:22:00,120 --> 00:22:00,920 it performance PR. 412 00:22:01,100 --> 00:22:05,200 You need that time to really go out and recruit tens, hundreds, maybe thousands 413 00:22:05,200 --> 00:22:09,800 of partners, depending upon your approach, and then really cultivate and manage 414 00:22:09,800 --> 00:22:10,640 those relationships. 415 00:22:10,740 --> 00:22:14,940 It's regular, weekly, monthly, quarterly communication with them to make sure 416 00:22:14,940 --> 00:22:16,560 that they haven't fallen off. 417 00:22:16,640 --> 00:22:17,980 They haven't forgotten about your brand. 418 00:22:18,100 --> 00:22:19,700 They haven't started promoting your competitor. 419 00:22:20,480 --> 00:22:23,920 I think that's why a lot of people find it too onerous to kind of manage in 420 00:22:23,920 --> 00:22:27,060 -house and they're like, hey, take this on for me, take it off my plate. 421 00:22:27,300 --> 00:22:27,740 Thank you. 422 00:22:28,260 --> 00:22:32,080 I think from a stepping back and thinking like, well, how do you set this up 423 00:22:32,080 --> 00:22:32,260 right? 424 00:22:32,320 --> 00:22:34,260 How do you see if this is a right test? 425 00:22:34,340 --> 00:22:38,200 We want to do as much of that kind of assessment upfront as we can through 426 00:22:38,200 --> 00:22:42,100 analysis of what data they have available through Google Analytics, through 427 00:22:42,100 --> 00:22:43,280 their other platforms. 428 00:22:43,720 --> 00:22:47,300 They have a live program and maybe it's not optimized, which is super common. 429 00:22:47,780 --> 00:22:48,620 Let's look at that. 430 00:22:48,740 --> 00:22:50,860 Let's take a look and kind of figure that out. 431 00:22:51,180 --> 00:22:53,700 What are the things that you see in a non-optimized program? 432 00:22:53,980 --> 00:22:57,400 What are the things that you look for where you're like, hey, look, these are 433 00:22:57,400 --> 00:23:00,880 the things that I want to look for to see if, hey, we can make some tweaks to 434 00:23:00,880 --> 00:23:03,500 really make this program a lot more incremental? 435 00:23:04,080 --> 00:23:08,000 I think not enough emphasis on full funnel. 436 00:23:08,600 --> 00:23:14,720 It sounds simple, but seeing kind of like the usual suspects from 2005 of like, 437 00:23:14,840 --> 00:23:16,540 okay, you got your cash back and you got your coupon. 438 00:23:16,720 --> 00:23:17,020 Great. 439 00:23:17,580 --> 00:23:18,400 Well, what the heck? 440 00:23:18,540 --> 00:23:19,240 What else is there? 441 00:23:19,280 --> 00:23:21,600 There should be a heck of a lot more. 442 00:23:21,780 --> 00:23:28,180 Your top 10 should have influencer, great, huge content publication network, 443 00:23:28,400 --> 00:23:31,220 maybe some like really niche provider. 444 00:23:31,820 --> 00:23:37,120 If you're in the baby space, you should have an insanely awesome baby blogger 445 00:23:37,120 --> 00:23:38,440 or network that's promoting you. 446 00:23:38,580 --> 00:23:41,160 That really nails your niche. 447 00:23:41,860 --> 00:23:45,700 There's all kinds of tech that's blown up in terms of like FinTech and card 448 00:23:45,700 --> 00:23:49,040 link offers where people are getting relevant suggestions when they go into 449 00:23:49,040 --> 00:23:52,580 things like Acorns and NerdWallet that are relevant to a lot of people. 450 00:23:53,160 --> 00:23:57,460 There's search providers that are willing to collaborate with you and hey, they 451 00:23:57,460 --> 00:24:01,420 may not be in your top 10, but they're a consideration to look at to diversify 452 00:24:01,420 --> 00:24:02,820 and push your competitors down. 453 00:24:03,260 --> 00:24:04,580 There's email providers. 454 00:24:05,160 --> 00:24:08,660 There's ways to tap media buyers that are sitting on the sidelines that are 455 00:24:08,660 --> 00:24:13,000 phenomenal at Google, Meta, TikTok that you can run on a pure CPA and get 456 00:24:13,000 --> 00:24:15,020 quality and not have to worry about fraud. 457 00:24:15,480 --> 00:24:18,320 All of them need management, all of them need handholding, all of them need 458 00:24:18,320 --> 00:24:19,280 rules and guidelines. 459 00:24:19,940 --> 00:24:23,860 But if you don't look into that top 10 and see a nice diversified portfolio, 460 00:24:24,120 --> 00:24:25,120 you're kind of missing the boat. 461 00:24:25,600 --> 00:24:25,740 Got it. 462 00:24:25,780 --> 00:24:30,160 So you sort of, what you are seeing is that there's probably too many brands 463 00:24:30,160 --> 00:24:35,500 think about affiliate as only being like the coupon code sites and not enough 464 00:24:35,500 --> 00:24:39,120 are thinking about the more the top of funnel, more awareness building ones. 465 00:24:39,220 --> 00:24:43,000 And those are the parts that are maybe even more incremental, but potentially 466 00:24:43,000 --> 00:24:44,060 more difficult to measure. 467 00:24:44,600 --> 00:24:47,420 Yeah, I think it's a combination of difficult to measure. 468 00:24:47,720 --> 00:24:51,680 Some of them do require some level of flat fee because they can command that 469 00:24:51,680 --> 00:24:53,380 for quality and for size. 470 00:24:53,560 --> 00:24:59,760 And so you want to, I think the common mishap is that it, they try to apply a 471 00:24:59,760 --> 00:25:04,120 paid search or paid social methodology to a channel that doesn't operate that 472 00:25:04,120 --> 00:25:04,380 way. 473 00:25:05,360 --> 00:25:10,220 And it just needs a lot more cultivation and handholding essentially. 474 00:25:10,480 --> 00:25:12,520 It's just, it's simply not programmatic yet. 475 00:25:12,700 --> 00:25:16,200 And I think people think that they try to apply programmatic principles to 476 00:25:16,200 --> 00:25:17,900 affiliate and that doesn't work. 477 00:25:18,260 --> 00:25:21,620 It needs to be actively managed, really actively managed. 478 00:25:22,100 --> 00:25:27,220 And so let's come back to like experimentation and how we might be able to do 479 00:25:27,220 --> 00:25:27,420 this. 480 00:25:27,460 --> 00:25:30,220 So like if I'm a brand and I want to run an experiment with these partners, is 481 00:25:30,220 --> 00:25:32,940 it literally just like picking up the phone and being like, Hey partner, we 482 00:25:32,940 --> 00:25:33,920 want to run this experiment. 483 00:25:34,160 --> 00:25:35,620 Like let's figure out how to go make it happen. 484 00:25:35,840 --> 00:25:40,200 Or what are the ways that you've seen this work tactically that a marketer 485 00:25:40,200 --> 00:25:42,480 today could potentially go and run with? 486 00:25:43,000 --> 00:25:45,020 Yeah, tactically and tactfully, right? 487 00:25:45,180 --> 00:25:51,000 It's like, it's a sensitive topic, you know, when a big brand reaches out to 488 00:25:51,000 --> 00:25:53,760 honey and it's like, Hey, we want to roll a holdout test. 489 00:25:53,880 --> 00:25:56,800 Like, I don't, I don't think honey's like jumping up and down saying, can't 490 00:25:56,800 --> 00:25:57,020 wait. 491 00:25:57,220 --> 00:26:00,820 I think there's a number of things that, you know, if you have the tracking 492 00:26:00,820 --> 00:26:05,300 dialed in really well, if you have no server to servers, a better methodology 493 00:26:05,300 --> 00:26:08,680 than pixel, if you have the attribution dialed in really well, Hey, go beyond 494 00:26:08,680 --> 00:26:09,280 last click. 495 00:26:09,280 --> 00:26:14,480 If you have your pricing dialed in really well, like being really smart about 496 00:26:14,480 --> 00:26:18,800 pricing, what you see on the data is higher quality and higher revenue and 497 00:26:18,800 --> 00:26:19,940 higher unified customer. 498 00:26:20,280 --> 00:26:22,520 You're already ahead of a lot of the players. 499 00:26:23,220 --> 00:26:27,220 And I think until you get to a certain size of volume, so you can run an 500 00:26:27,220 --> 00:26:29,820 appropriate test, you know it better than anyone in MMM. 501 00:26:30,040 --> 00:26:31,700 You've got to have that sample size. 502 00:26:31,760 --> 00:26:35,860 You've got to have that data set until you get there probably doesn't make 503 00:26:35,860 --> 00:26:40,060 sense to approach a loyalty partner and say, Hey, I want to run a test, a geo 504 00:26:40,060 --> 00:26:40,820 or a holdout. 505 00:26:41,680 --> 00:26:45,040 Now, good news is for those that are more sophisticated, have gone through 506 00:26:45,040 --> 00:26:50,240 those steps and have kind of gotten to like PhD level affiliate marketing, you 507 00:26:50,240 --> 00:26:55,040 know, then it's time to approach the partners that you have questions about and 508 00:26:55,040 --> 00:26:57,780 say, Hey, we want to run an experiment here. 509 00:26:57,940 --> 00:26:59,420 Would you be open to it? 510 00:26:59,780 --> 00:27:01,460 Here's how we're kind of thinking about it. 511 00:27:01,540 --> 00:27:02,800 We'd like to hear your feedback. 512 00:27:03,460 --> 00:27:04,480 And it's a relationship. 513 00:27:04,840 --> 00:27:05,720 It's a two way street. 514 00:27:05,920 --> 00:27:06,700 It's a negotiation. 515 00:27:07,380 --> 00:27:08,820 It comes in ebbs and flows. 516 00:27:08,920 --> 00:27:09,420 I'll be honest. 517 00:27:09,520 --> 00:27:12,640 Some partners in that space are like, we can get this revenue elsewhere. 518 00:27:12,940 --> 00:27:13,780 Good day, sir. 519 00:27:13,900 --> 00:27:14,980 And they move on. 520 00:27:15,200 --> 00:27:17,520 And I think that that's really interesting. 521 00:27:18,180 --> 00:27:22,160 I think that the better partners are going to be the ones that are, you know, 522 00:27:22,220 --> 00:27:27,080 in the right use cases are willing to make testing a little bit easier and 523 00:27:27,080 --> 00:27:32,400 stand behind their offering with confidence and allow the brand to make the 524 00:27:32,400 --> 00:27:35,420 determination based on it, assuming that the methodology is right. 525 00:27:35,920 --> 00:27:39,560 I think where people get into trouble is when they throw the baby out with the 526 00:27:39,560 --> 00:27:41,600 bathwater and say, Oh, coupon and loyalty don't work. 527 00:27:41,720 --> 00:27:43,260 It's just a nonstarter for me. 528 00:27:43,700 --> 00:27:49,860 I think it's ultimately more about how you value that partner than throwing it 529 00:27:49,860 --> 00:27:50,260 out completely. 530 00:27:50,500 --> 00:27:53,120 Are there examples where you need to remove and move on? 531 00:27:53,240 --> 00:27:53,720 Absolutely. 532 00:27:54,160 --> 00:27:58,800 But I think it's all more about the right, accurate tracking of value, which, 533 00:27:58,920 --> 00:28:02,160 which is obviously what you guys are trying to do with what you're building. 534 00:28:02,580 --> 00:28:04,500 Yeah, I think that's right. 535 00:28:04,600 --> 00:28:08,320 And I think it's just it's such a it's such a tricky and hard problem. 536 00:28:08,480 --> 00:28:11,600 And I mean, at Recast, we spend a lot of time on affiliate because affiliate is 537 00:28:11,600 --> 00:28:15,240 really problematic, actually, for doing marketing mix modeling. 538 00:28:15,440 --> 00:28:22,780 And the reason why is that in general with affiliate programs, you pay for the 539 00:28:22,780 --> 00:28:27,400 marketing activity after the conversion has already happened, which is 540 00:28:27,400 --> 00:28:31,020 different from how all other marketing works, where you spend money and you get 541 00:28:31,020 --> 00:28:33,040 impressions and then conversions happen later. 542 00:28:33,480 --> 00:28:38,220 In affiliate, it's reversed where you get the revenue in the door, and then the 543 00:28:38,220 --> 00:28:40,020 spend happens for the affiliate partner. 544 00:28:40,620 --> 00:28:44,480 And if you think about the way that like all of MMM is structured to work, it 545 00:28:44,480 --> 00:28:48,200 sort of is implicitly making the assumption that the spend is happening and 546 00:28:48,200 --> 00:28:51,260 then conversions are happening later, not the opposite. 547 00:28:52,180 --> 00:28:56,720 And so MMM modeling is really, really hard to do correctly with affiliates. 548 00:28:56,760 --> 00:29:00,960 And in fact, when we first started building Recast, what we found is that the 549 00:29:00,960 --> 00:29:03,560 model was like too smart, it was too good. 550 00:29:03,960 --> 00:29:07,860 When we would include affiliate activity in the model, the model basically just 551 00:29:07,860 --> 00:29:11,100 found out what is the payment for affiliate conversions. 552 00:29:11,260 --> 00:29:14,100 And we were the model basically just identified, hey, look, you're paying $5 553 00:29:14,100 --> 00:29:14,940 for every conversion. 554 00:29:15,480 --> 00:29:18,420 And so that actually doesn't help because then it's not truly measuring 555 00:29:18,420 --> 00:29:19,100 incrementality. 556 00:29:19,140 --> 00:29:23,700 It's just finding the relationship in the data, which is that when your spend 557 00:29:23,700 --> 00:29:28,300 goes up by X amount, your revenue goes up by some 20X that amount, because 558 00:29:28,300 --> 00:29:31,400 there's a 5X return on investment on that affiliate spend. 559 00:29:31,940 --> 00:29:32,540 So it's tricky. 560 00:29:32,800 --> 00:29:33,500 It's hard. 561 00:29:33,600 --> 00:29:38,040 It's a really hard problem because of that closed loop system. 562 00:29:38,840 --> 00:29:42,520 And so we have spent a bunch of time at Recast really thinking about how do we 563 00:29:42,520 --> 00:29:47,180 break that connection and make it so that the model isn't just going to find, 564 00:29:47,540 --> 00:29:50,960 hey, what's the affiliate relationship here, but actually thinking about how do 565 00:29:50,960 --> 00:29:54,200 we understand the relationship between other marketing channels, the 566 00:29:54,200 --> 00:29:58,420 interactions with affiliate, and then what affiliate is going to happen, like 567 00:29:58,420 --> 00:30:01,540 what affiliate spend is going to happen no matter what, how much of that is 568 00:30:01,540 --> 00:30:01,840 incremental. 569 00:30:02,300 --> 00:30:04,960 But it's a very, very, very hard problem. 570 00:30:05,540 --> 00:30:10,060 And so a lot of MMM researchers will just not include affiliate at all because 571 00:30:10,060 --> 00:30:10,980 they're like, it's too hard. 572 00:30:11,060 --> 00:30:12,020 It messes up the model. 573 00:30:12,340 --> 00:30:14,220 It's really a tricky problem. 574 00:30:14,680 --> 00:30:20,820 And I think until we as an industry figure out better ways to get really smart 575 00:30:20,820 --> 00:30:24,520 about measuring affiliate, and I'd love to see people developing more tools for 576 00:30:24,520 --> 00:30:28,560 doing this sort of holdout testing in affiliate channels. 577 00:30:28,920 --> 00:30:33,940 I think it's just really tough for marketers to get a good solid measurement 578 00:30:33,940 --> 00:30:35,660 that they have a lot of confidence in. 579 00:30:35,980 --> 00:30:39,140 So as you said, they're sort of stuck piecing the different pieces together 580 00:30:39,140 --> 00:30:40,760 from the different evidence that they have. 581 00:30:41,420 --> 00:30:43,340 And so, I don't know, I'm really excited about the future. 582 00:30:43,780 --> 00:30:48,020 But what I want to see is I want to see more of these like experimentation type 583 00:30:48,020 --> 00:30:49,640 tools being developed in the industry. 584 00:30:50,020 --> 00:30:53,800 So that way, if I'm a brand and I want to experiment with honey, I can easily 585 00:30:53,800 --> 00:30:54,400 run an experiment. 586 00:30:54,540 --> 00:30:56,120 I don't just have to take their word for it. 587 00:30:56,200 --> 00:30:58,280 I can actually say like, okay, look, there's a holdout test. 588 00:30:58,400 --> 00:31:00,980 We're going to have honey in half the country and not in the other half and 589 00:31:00,980 --> 00:31:02,080 really see what happens. 590 00:31:03,060 --> 00:31:04,340 Yeah, I think it's possible. 591 00:31:04,580 --> 00:31:06,100 I think it needs to be more embraced. 592 00:31:06,220 --> 00:31:07,440 I 100% agree with you. 593 00:31:07,640 --> 00:31:09,160 An interesting thought came into my head. 594 00:31:09,220 --> 00:31:14,320 I'm not sure if it would work, but there is a percentage of affiliate that does 595 00:31:14,320 --> 00:31:15,880 do some of that upfront payment. 596 00:31:16,300 --> 00:31:20,800 There is a percentage of affiliate that is, which is kind of debatable whether 597 00:31:20,800 --> 00:31:21,900 that's affiliate or not. 598 00:31:21,900 --> 00:31:24,520 But it's there and it's happening. 599 00:31:24,920 --> 00:31:30,100 And so some will move to cost per click models, maybe to avoid certain legal 600 00:31:30,100 --> 00:31:32,020 issues or constraints. 601 00:31:32,780 --> 00:31:37,720 So let's say an MMM candidate brand working with Recast were to say, hey, look, 602 00:31:38,040 --> 00:31:41,760 we're going to run a certain amount of our, like influencers is a good example. 603 00:31:42,080 --> 00:31:46,860 A larger percent of that is like, hey, payment upfront for a multi-staged 604 00:31:46,860 --> 00:31:49,160 campaign, let's say multiple posts, et cetera. 605 00:31:49,700 --> 00:31:53,600 What if maybe starting with a slice of affiliate, like an, like Instagram 606 00:31:53,600 --> 00:31:58,860 influencers for a retailer, if we were to say, okay, we're going to, how much 607 00:31:58,860 --> 00:32:03,700 data would you think you would need for a type of brand to run? 608 00:32:04,000 --> 00:32:09,200 Like maybe looking back over the course of a year, I'm just kind of teeing up a 609 00:32:09,200 --> 00:32:09,640 case. 610 00:32:09,840 --> 00:32:15,200 Do you think you could learn about the incrementality of that influencer 611 00:32:15,200 --> 00:32:15,800 strategy? 612 00:32:16,280 --> 00:32:16,400 Totally. 613 00:32:16,540 --> 00:32:17,880 If there's enough spent, right. 614 00:32:17,900 --> 00:32:18,940 And this is the thing that we've done. 615 00:32:19,200 --> 00:32:22,140 And again, this is a thing basically like when we're working with these brands 616 00:32:22,140 --> 00:32:26,360 and they have, they might have influencers, some of which are sort of 617 00:32:26,360 --> 00:32:29,720 traditional affiliate where you're paying per conversion and some of which are 618 00:32:29,720 --> 00:32:30,260 pay upfront. 619 00:32:30,440 --> 00:32:33,640 And we'll basically split those in two and we'll treat them differently in the 620 00:32:33,640 --> 00:32:36,260 statistical model because you have to, because of the problems I was just 621 00:32:36,260 --> 00:32:36,900 talking about. 622 00:32:37,560 --> 00:32:42,340 And so with that spend, right, the paid upfront influencers, we can treat that 623 00:32:42,340 --> 00:32:45,200 like any other marketing channel because you're paying money, you're getting 624 00:32:45,200 --> 00:32:48,280 impressions and then conversions are happening. 625 00:32:49,080 --> 00:32:51,740 And if you've been running an affiliate program for a year and it's a 626 00:32:51,740 --> 00:32:55,680 substantial amount of spend, yes, we can absolutely identify the incrementality 627 00:32:55,680 --> 00:32:56,120 of that. 628 00:32:56,340 --> 00:32:59,440 And we've done that with a bunch of our partners in terms of being able to say, 629 00:32:59,560 --> 00:33:03,280 look, when you are investing more into these influencers, you're driving in 630 00:33:03,280 --> 00:33:06,600 number of additional dollars of revenue or in number of additional conversions. 631 00:33:07,100 --> 00:33:11,680 And therefore we can back into what the incremental return on investment or 632 00:33:11,680 --> 00:33:15,060 cost per acquisition is from that investment that you're doing. 633 00:33:15,340 --> 00:33:18,300 And that's actually like one of our key selling points at Recast is that we can 634 00:33:18,300 --> 00:33:22,260 give you insight into channels like that, that otherwise are very difficult to 635 00:33:22,260 --> 00:33:26,060 measure because as we all know, it's difficult to parse apart. 636 00:33:26,300 --> 00:33:29,060 Like if you're running influencers on YouTube, right? 637 00:33:29,460 --> 00:33:31,360 Is it your YouTube ads that are happening? 638 00:33:31,540 --> 00:33:32,280 Is it the influencer? 639 00:33:32,820 --> 00:33:35,600 A lot of times those people aren't clicking on any link or actually engaging 640 00:33:35,600 --> 00:33:39,920 with that post in any way that would lead them directly to your website, but 641 00:33:39,920 --> 00:33:43,460 they are, you know, maybe they're watching YouTube on their TV and then they're 642 00:33:43,460 --> 00:33:45,600 going on their phone and searching for your brand. 643 00:33:45,820 --> 00:33:48,360 You really need to have a good way of actually being able to understand that 644 00:33:48,360 --> 00:33:52,380 connection, even if they're not necessarily using the coupon code or the vanity 645 00:33:52,380 --> 00:33:56,140 URL, which I personally almost never use. 646 00:33:56,240 --> 00:34:00,620 And so like I can empathize with those buyers that don't necessarily do that. 647 00:34:01,000 --> 00:34:04,220 And so you need to understand what are the sort of statistical relationships in 648 00:34:04,220 --> 00:34:06,320 the data in order to measure a channel like that effectively. 649 00:34:06,920 --> 00:34:11,139 How much does the payment upfront improve your fidelity in your opinion in that 650 00:34:11,139 --> 00:34:11,540 factor? 651 00:34:11,699 --> 00:34:14,780 Or is it like number three on a series of factors? 652 00:34:15,219 --> 00:34:19,540 So it definitely helps from an econometric modeling perspective. 653 00:34:19,760 --> 00:34:21,020 It definitely helps, right? 654 00:34:21,060 --> 00:34:27,179 Because you don't have that problem of the sort of circular causality thing 655 00:34:27,179 --> 00:34:29,820 where you're paying after the fact of conversion. 656 00:34:30,040 --> 00:34:35,219 And so from a causal modeling perspective or a causal inference perspective, 657 00:34:35,219 --> 00:34:38,260 it's definitely a lot easier to get a read in that situation. 658 00:34:39,179 --> 00:34:44,980 For the ones that are done just on a paper performance basis, we definitely 659 00:34:44,980 --> 00:34:48,940 have a lot more sort of asterisks in terms of thinking about, hey, is this 660 00:34:48,940 --> 00:34:51,480 measure actually as accurate as we want? 661 00:34:51,800 --> 00:34:54,380 And then a lot of times that's the point where we're talking to these brands 662 00:34:54,380 --> 00:34:57,080 and saying like, look, we should figure out how can we go and test this channel 663 00:34:57,080 --> 00:35:00,560 to make sure that we're getting a good read here or get outside information 664 00:35:00,560 --> 00:35:03,700 about the true incrementality so we can use it for real decision making. 665 00:35:04,080 --> 00:35:07,960 Just to play out the scenario, not to like over dwell on it, but like if it 666 00:35:07,960 --> 00:35:13,140 was, let's say a brand was spending 500 grand a month on affiliate as a 667 00:35:13,140 --> 00:35:19,140 channel, let's say you had a 12 month look back, assuming a relative 50 cent 668 00:35:19,140 --> 00:35:20,980 cost per click, we can do the quick math, right? 669 00:35:21,000 --> 00:35:22,740 We can just model it out today on the call. 670 00:35:23,500 --> 00:35:24,560 I'm just kidding. 671 00:35:25,060 --> 00:35:27,600 Do you think you can get a read? 672 00:35:27,700 --> 00:35:29,600 I mean, again, it depends on what else they're doing, right? 673 00:35:29,640 --> 00:35:30,780 But that's a fair amount of spend. 674 00:35:31,040 --> 00:35:34,500 You should definitely be able to start to model out what's the relationship 675 00:35:34,500 --> 00:35:34,900 there. 676 00:35:35,340 --> 00:35:35,540 Yeah. 677 00:35:35,660 --> 00:35:39,600 You got to be making a lot of money through that lever to be spending that 678 00:35:39,600 --> 00:35:44,560 much, which is probably not a lot of retail, not a lot of DTC income, but 679 00:35:44,560 --> 00:35:45,160 there's some. 680 00:35:45,580 --> 00:35:46,720 You had a 12 month look back. 681 00:35:47,000 --> 00:35:50,560 The question is how much of that would you need to be paying on a cost per 682 00:35:50,560 --> 00:35:53,140 click or the opposite of an affiliate model? 683 00:35:53,940 --> 00:35:57,860 So it's like, does it justify switching over to a cost per click or upfront 684 00:35:57,860 --> 00:35:58,420 payment? 685 00:35:58,760 --> 00:36:00,620 I would encourage brands to experiment, right? 686 00:36:00,680 --> 00:36:03,480 Like, can we carve out a small part of that and switch it over and see what 687 00:36:03,480 --> 00:36:04,580 happens, right? 688 00:36:04,640 --> 00:36:06,560 Let's take a test and learn. 689 00:36:06,720 --> 00:36:10,060 I mean, this is our view on like literally every problem, which is like, let's 690 00:36:10,060 --> 00:36:10,560 run an experiment. 691 00:36:10,700 --> 00:36:11,800 Let's take a test and learn approach. 692 00:36:11,880 --> 00:36:15,760 Let's figure out how can we test this and validate these assumptions without, 693 00:36:15,940 --> 00:36:19,060 and we don't necessarily need to be like, let's make a whole big change for 694 00:36:19,060 --> 00:36:20,740 this whole program all at once. 695 00:36:20,940 --> 00:36:24,220 One of the small things that we can do to start getting learnings and that 696 00:36:24,220 --> 00:36:27,860 could inform what the next step, we don't have to plan out the next two years 697 00:36:27,860 --> 00:36:28,700 of our marketing spend. 698 00:36:29,100 --> 00:36:33,080 We can say, let's run an experiment over the next two months and then take that 699 00:36:33,080 --> 00:36:36,960 and use that to make a decision about what we do three or four months from now. 700 00:36:37,560 --> 00:36:38,120 I love that. 701 00:36:38,260 --> 00:36:41,340 It's totally in line with our philosophy and always be testing and 702 00:36:41,340 --> 00:36:41,860 experimentation. 703 00:36:42,280 --> 00:36:45,940 I think that there's so much alignment there and taking it like one partner at 704 00:36:45,940 --> 00:36:49,960 a time, one geo at a time, breaking down the problem, the first principles. 705 00:36:50,160 --> 00:36:54,580 I think that's really something that is there for people to try and be willing 706 00:36:54,580 --> 00:36:55,860 to test and learn from.