Subscribe
Share
Share
Embed
Your guided tour of the world of growth, performance marketing, customer acquisition, paid media, and affiliate marketing.
We talk with industry experts and discuss experiments and their learnings in growth, marketing, and life.
Time to nerd out, check your biases at the door, and have some fun talking about data-driven growth and lessons learned!
Welcome to another edition of the Always Be Testing podcast with your host Tye De
Grange.
Get a guided tour of the world of growth, performance marketing, customer
acquisition, paid media, and affiliate marketing.
We talk with industry experts and discuss experiments and their learnings in
growth, marketing, and life.
Time to nerd out, check your biases at the door, and have some fun talking
about data-driven growth and lessons learned.
Hello.
Welcome to the Always Be Testing podcast.
I am absolutely thrilled to have Michael Kaminsky on as our guest.
Always Be Testing podcast, where we talk about growth, performance marketing,
testing, experimentation, learning, partner marketing, and all those things.
And yeah, I'm thrilled to have Michael on today.
Michael Kaminsky, the CEO of Recast, a modern approach to multimedia mixed
modeling and all things attribution and incrementality.
Welcome.
Thanks for having me, Ty.
I am very excited to be here, excited for our discussion today.
Absolutely, absolutely.
It's going to be a good one and I'm excited to dive in.
This is, I feel like, such a hot button topic.
For years, people have debated the topics of media mixed modeling,
incrementality.
It's a hot button topic.
It's a topic that comes up for growth people, for data science nerds, for paid
marketers, for affiliate and partner marketers.
And you are the guy to talk to because you've launched a business that solely
focuses on this.
So let's just jump in.
What the heck is media mixed modeling?
Why is it so hot right now?
What's going on?
Yeah, great question.
So I think everyone who's listening to this podcast can be familiar with a
bunch of the changes that have been happening in the industry over the last
couple of years, largely related to reduced ability to track people across the
internet because of changes with app tracking transparency from Apple that
rolled out with iOS 14.5, more changes to the ability to track that are coming
with iOS 17, plus increased use of ad blockers, privacy regulations like GDPR.
It has made the ability to track people across the internet consistently much,
much, much harder.
And so that means that a lot of the tools that a lot of marketers, and
especially digital-focused marketers, got used to using over the last five or
10 years, all of the digital tracking tools, first-touch attribution, last
-touch attribution, all of those sorts of tools, they're now less reliable.
And marketers know that, and they're starting to see the flaws in those
methodologies that have really been made much more acute in the last couple of
years with the changes to tracking.
And so marketers have started to get really interested in, what are the
alternative ways of measuring marketing performance that doesn't just rely on
digital tracking?
And marketing mix modeling is one of those forms of measuring marketing
performance.
It's basically an econometric model that looks at patterns in the data to try
to understand the true incrementality of marketing performance.
And because marketers no longer feel that they can just rely on digital
tracking, it's becoming a thing that a lot of people are talking about.
That being said, because marketing mix modeling, a lot of the old ways of doing
it were not really built for modern digital marketing, there's been a lot of
technologies that have been developed in the last couple of years of people
trying to figure out, how do we make marketing mix modeling actually work for
the problems that marketers, modern marketers face today?
And that's what we're doing at Recast.
And that's why everyone is sort of talking about this and trying to figure out
how do we actually make this tool work for us with all of the problems that
we're facing today in 2023 as marketers.
That's awesome.
Yeah, it certainly seems to be insane demand for the right type of brand, the
right type of situation.
Just thinking back, when did this all start?
The notion of media mix modeling has been around for quite a long time.
Yeah, that's exactly right.
So it's a really interesting history.
So media mix modeling, marketing mix modeling has been around almost as long as
computers have been around.
If you think back to pre-e-commerce, pre-internet days, and you're the CMO of a
CPG company like Pepsi or Gillette or whatever, every year you need to make
decisions about how you're going to allocate your marketing budget across the
different ways that you can spend it.
And that might be with doing promotional activity or running TV ads or running
print ads or running radio ads.
And there's no digital tracking at all.
People don't buy online.
The internet doesn't really exist, at least not the way that it exists today.
And so how are you going to make that decision?
And the way that CMOs made that decision historically, especially in fast
moving consumer goods industries, was through these econometric projects.
Effectively, you would hire a statistician or an econometrician.
They would look at your historical data.
At times when you spent more on TV, how many additional Pepsis or Razors did
you sell?
At times when you spend more on radio, how many additional Pepsis or Razors do
you sell?
And so these econometricians would do these research projects once a year.
They were very expensive.
These were highly trained PhDs.
They were based on consulting projects.
They would come in.
They would do this analysis.
Lots and lots of research and investigation goes into it.
They produce a report.
And then the CMO looks at it and says, OK, great.
We're going to allocate $10 million to TV and $15 million to radio and $5
million to print.
And then the marketers would go out and buy that media at upfronts once a year.
And then the marketing team would go out and sort of execute on that plan with
all of the media that they had bought.
And so that's where this idea came from, is how do we measure marketing
effectiveness when we don't track anyone at all?
We don't have any digital tracking to rely on.
And so MMM, media mixed modeling, has been around for 50 or 60 years as these
types of fast-moving consumer good brands have needed a way to measure
marketing effectiveness without being able to digitally track someone.
Over the last 10 or 15 years, it sort of fell out of favor a little bit,
especially with the rise of e-commerce, where we could track people.
But now that our ability to track is again reduced, it's coming back into
favor.
But we all recognize that the old way of doing it, where you hire a bunch of
statisticians and they produce a report once every six months or once a year,
doesn't really match the way that modern marketing actually works.
That's amazing.
Yeah, it's almost come full circle, it sounds like, right?
Yeah, totally.
Well, I mean, what's old is new again, right?
And this is the way that marketing gets to work.
But I think the key thing here is that we have to figure out what are we
actually trying to do with this technology?
And then how can we get it into the hands of more marketers so they can
actually use it to make real decisions?
That's what the core thing is that we're trying to do here, is how do we
actually drive businesses forward by helping them accurately measure their
marketing effectiveness?
That's the most important thing.
And when we're thinking about MarTech tools, any sort of attribution
methodology, we always want to be thinking about how are we using this to drive
the business forward?
How are we using it to actually estimate incrementality so that we can spend
our dollars where they're most effective?
And is incrementality like the kind of big value that really is derived from
MMM?
So in my view, incrementality should be really the only thing that matters for
marketers.
Marketers should be focused on how are we measuring incrementality for our
business?
And let's maybe take a step back for the listeners.
What do we mean when we say incrementality?
For me, incrementality means if I spend an additional dollar in this marketing
channel, how much additional revenue is that dollar going to drive?
Or if I pull a dollar out of some marketing channel, how much revenue are we
going to lose from having pulled that dollar out of that channel?
That's what incrementality means to us.
It is the causal relationship between the marketing activity that we're doing
and then the business results on the other side.
And that's the thing that we want to understand.
Because if we understand that, if we understand true incrementality, then we
can actually optimize our marketing budget.
Because we can take our whole marketing budget and then allocate the dollars to
where they're all most effective.
And that's the best that we can possibly do as marketers.
Of course, there's a bunch of complications that go around that.
But in the ideal scenario, that's what we're aiming for when we're talking
about doing marketing measurement.
And so incrementality is the thing.
Everyone should always only be thinking about incrementality.
In all of these different measurement methods, we should be thinking about is
this getting us closer to incrementality or not?
Or this measurement method works in this case, but in this other case, it's not
actually good at measuring incrementality.
And so that's what I want marketers to be thinking about is how is this
measuring incrementality and how is it getting us closer to that ideal state of
being able to really understand the true causal relationships between our
marketing activity and our business performance.
So incrementality, hugely important concept.
MMM is one way of measuring incrementality.
And if you do MMM right and you're thinking about it the right way, you should
be trying to use MMM to measure incrementality the same way that you should be
using experimentation to try to measure incrementality.
The same way that you should be thinking about how does our last touch
attribution not measure incrementality?
And in what cases should we not be relying on this tracking methodology to
actually understand the true incrementality of what we're doing?
All of those are the sorts of questions that I think that marketers should
really be focused on.
Yeah, I love that.
And I love the concept of like, hey, if this campaign marketing lever were to
be removed, would you have received that conversion event or value or revenue
without it?
Exactly.
It's a good kind of framework as well to build on your definition of
incrementality.
And honestly, I'm really excited to talk with you, Tai, because I know that
you're an expert in like the affiliate marketing space.
And the affiliate marketing space is one, I think like a lot of people don't
really understand the full range of affiliate marketing and what's possible
with affiliate marketing.
But also, it's sort of famously a really difficult channel to measure the
incrementality of.
This is a thing that we've spent a lot of time thinking about at Recast.
I know you have spent a lot of time thinking about it.
So I'm really excited to have this conversation today because I want to ask you
about what are the best practices around measuring the true incrementality of
these affiliate programs?
How can marketers get smart about that in a world where it just it feels like
it's a lot more difficult to test for a bunch of different reasons, a bunch of
tricky complications that go into it.
So maybe I guess like, I'd love to hear from you, Tai.
What are the main things that people don't understand about affiliate marketing
and maybe talk a little bit about like, why is it so hard to test these
affiliate channels?
I see what you did there, Mike.
You're flipping it on the interviewer interviewing me.
I get it.
Okay.
I want to get something out of this conversation too.
This is for me.
Absolutely.
Yeah, it's so funny because I feel like affiliate and I've been sharing this a
lot is the most misunderstood performance marketing lever and likely the most
underrated as a result of that.
A lot of the misconceptions are also stem around the topic of incrementality
and other related topics.
Historically, brands have tried to sort through the challenges of fraud.
I think that's common for any performance marketing channel and digital
channels, especially in their early days as they matured, that fraud typically
got better tools caught up.
People got caught up in how to catch it and improve it.
You had misaligned network incentives from the 2000s and 2010s where they were
charging high rates.
They could command higher rates.
Sometimes they were aligned with paying out partners more rather than paying
out what was efficient and accurate and rewarded and valued in terms of what
was tracked.
There's a lot of things that surround the affiliate space that get
misconstrued, misunderstood, that are historical, that are real, that are
current, that are perception versus reality.
You've got coupon and deal sites.
The theory and some of the perception is that that's what they are.
That's what affiliate is.
They're simply taking credit at the last minute.
There's some truth to that, but that's certainly not what affiliate all is.
When I think about affiliate, I definitely think about these coupon sites.
Whenever I'm making a purchase online, when I'm at the checkout, I always
Google coupon and I go to a coupon site and I grab the coupon.
I totally see that case as being like, look, there's a good chance this isn't
truly incremental because I'm going to buy whether I get the coupon or not, but
if I get the extra 15%, great.
What are the other types of affiliate that people maybe aren't thinking about?
I mean, the reality is under partner marketing, under affiliate marketing,
under influencer marketing, I consider them all the same.
Fundamentally, when it comes to the work and the scope and the specifics of the
actions we do for our clients, it's affiliate and influencer are different, but
fundamentally they're the same.
When you look at affiliate marketing, it's a way for you to reach people multi
-channel across multiple touch points of Google, Meta, TikTok, review sites,
gift guides, media houses.
Think apartment therapy, BuzzFeed, Sports Illustrated, American Express,
rewards.
There's just thousands of quality content touch points to reach consumers via
affiliate and affiliate in its definition really is saying, hey, I want to pay
for outcomes.
I want to pay for valued outcomes, not just clicks and eyeballs.
You think about Meta and Google, essentially that's the payment mechanism that
you're opting into.
Yes, you're backing out to a valued ROI, ROAS, MER, CPA, whatever that KPI
might be for your business.
But with affiliate, you're able to say for a much larger percentage of the
action and a much larger percentage of the budget, I can pay for an outcome and
say, I need to pay a $20 CPA.
I need to pay $5 a lead, whatever that might be for the brand.
Totally.
Okay.
That makes sense.
Paying for outcomes.
I think it's easier for me to stick case that say like, look, an affiliate deal
with BuzzFeed could be hugely valuable for a brand when the honey toolbar thing
is maybe not.
But how do we think about measuring that?
How do we prove that as a marketer?
If I'm an affiliate marketer, how do I go prove that to my boss?
I think this is a thing that I've spent a bunch of time thinking about.
It just feels very hard to test this sort of channel.
With a Facebook, we can show ads to some people and not to others, or we can
run Facebook ads in Washington, but not California, and you can start to think
about running experiments.
But with affiliate, that seems a lot harder.
How do you think about actually doing that to prove the value of this marketing
activity?
Yeah, you don't really have the closed loop ecosystem of meta or Google.
You do have a network, like an impact that'll track all your partner activity
for the most part.
That's a win and that's an opportunity to kind of look into the data and really
dive into that.
But you're right.
There are challenges with it.
It's not as impossible as some might think, though.
I think that's a big call out.
Don't lose hope.
We'll kind of dive in and work through it together here and figure it out.
But the reasons why it's hard are long and numerous.
It's not an easy thing.
I think the toolbars you referenced are a big part of that.
Historically, that's been debated for probably about 20 years.
There needs to be, I think, more...
In a matter of fact, just at a macro level, the performance marketing agency
I'm part of is running studies and really exploring a lot of this and
resurfacing the topic of toolbars like Honey to say, okay, what type of data do
people have access to transparently to see how much traffic came from a toolbar
-like partner or browser plugin versus maybe other traffic sources that partner
can provide.
Ultimately, we are very cautious.
We are very wary.
We take a very judicious approach.
If we work with a partner like that, sometimes we inherit a partner like that
when a client has it already in their program.
And we want to look at a lot of different data sources to determine if this
makes sense for their overall strategy.
A lot of things are kind of required to think about in there.
What stage are they in?
Are they in a conquesting stage?
Are they in an aggressive spend stage?
Are they trying to be very careful about profitability and maybe they've
reached a maturity level or level of brand awareness?
Those are important macro factors that play into if you would want to leverage
a toolbar at all.
For us, it makes it even more complicated when a lot of brands are not yet
adopting anything beyond a last-click attribution model.
That really hamstrings the efforts to really accurately...
Obviously, this is different than incrementality, but it's related.
You have this vast improvement in content, this vast improvement of quality,
yet you're not attributing value to more than just the last click.
What a missed opportunity for brands in the affiliate marketing space.
And so I think that's a really important piece of this.
That's next step piece that people need to get to in terms of figuring out and
making sense of this before they can really proceed with, I think, the
incrementality question.
Just having the baseline stuff right is surprisingly not there.
Not surprising to me, given how I've seen the data and marketing measurement
setups of a lot of brands.
I'm curious, tell me about some of these studies that you have run with these
different affiliates.
Maybe it's like, hey...
I don't know what you've run, so I'm really curious to hear some examples of
this in the wild.
But have you worked with a brand that's using a toolbar and then you're like,
let's turn off the toolbar for a month and see what happens?
What are the different flavors of that that you run and what are the results
that you've seen?
Yeah, we've seen a number, both from an in-house as a marketer, in-house at
large companies, as an agency, and talking to folks closely in the industry.
We've seen a number of things.
And I think the challenge is, oftentimes, it's hard to get a true read because
running a true holdout is often, as you know, one of the best ways when you're
not doing MMM to be able to measure for incrementality.
And so as a result, when you don't control the environment like you do on Meta
and Google, that really becomes harder geographically from a time perspective.
Some brands will do kind of like a before and after.
And as you know, that's inherently flawed.
So I think what often happens is people get false positives or false negatives
because they're not really understanding the science around incrementality and
how you need to really look at that test in a very clean way.
And so I have seen a movement towards more and more willingness and
collaboration to run holdout tests.
The challenge comes in the fact that a lot of partners, it requires partner
participation, partner collaboration, the partner that is essentially at risk
of losing out on a partnership, typically with a larger brand, because those
are the brands that have the incentive, time and effort, money and budget to
actually run a test like this.
So it's really interesting.
I do think it does require...
So what often happens in the lead up to becoming that big brand is often we
apply workarounds as best we can, like really effective pricing, looking at
cohort analysis over time to see, hey, what quality type, what new customer,
new to file customer data came through to a particular brand.
Maybe we do try to turn off strategy and kind of see.
And there's a surprising number of brands that actually will say, let's turn it
back on based on their internal data that they'll see.
So it's not necessarily just all full hearty people chasing dumb money.
There is a level of brands that are saying, hey, we do want to opt back into
this, whether that's maybe it's conquesting vis-a-vis customers.
But I think to get to true MMM level, true data science level incrementality, I
think we have to be smarter about running more holdout tests.
I think the industry still has a ways to go.
But I do believe that that's been the most effective I've seen.
Let's say that I'm a brand and I've got a normal mix right now.
So it's a lot of Facebook, a lot of digital marketing.
Maybe I've got some offline stuff.
And I'm interested, like, okay, affiliate might be a new growth path for us.
What would you recommend that I do as a marketer at this brand to go test
affiliate and figure out if it's going to work for us?
Totally.
I have a checklist of things, both published and in my brain with our team that
we go through.
I think I can run through that real quickly and then jump into what's there.
But I think the obvious product market fit, obvious level of revenue needs to
be there.
Typically, a level of retention in the client, that flat retention curve that
we talk about in growth, that folks are coming back, there's value there.
There's not crazy high return levels or issues with that.
The healthy conversion is important in affiliate because if you think about
affiliate, they are putting skin in the game, time, money, expertise to promote
your brand.
And then expecting that return in the form of the commission as opposed to,
hey, I'm getting money up front.
Yes, both happen, but more often on the commission side in the affiliate space.
So as a result, that conversion rate needs to be healthy.
You go through that checklist top to bottom and say, hey, let's go forth and
conquer.
Time is important too.
We need to be able to give this a real, it's relationship-based, it's some call
it performance PR.
You need that time to really go out and recruit tens, hundreds, maybe thousands
of partners, depending upon your approach, and then really cultivate and manage
those relationships.
It's regular, weekly, monthly, quarterly communication with them to make sure
that they haven't fallen off.
They haven't forgotten about your brand.
They haven't started promoting your competitor.
I think that's why a lot of people find it too onerous to kind of manage in
-house and they're like, hey, take this on for me, take it off my plate.
Thank you.
I think from a stepping back and thinking like, well, how do you set this up
right?
How do you see if this is a right test?
We want to do as much of that kind of assessment upfront as we can through
analysis of what data they have available through Google Analytics, through
their other platforms.
They have a live program and maybe it's not optimized, which is super common.
Let's look at that.
Let's take a look and kind of figure that out.
What are the things that you see in a non-optimized program?
What are the things that you look for where you're like, hey, look, these are
the things that I want to look for to see if, hey, we can make some tweaks to
really make this program a lot more incremental?
I think not enough emphasis on full funnel.
It sounds simple, but seeing kind of like the usual suspects from 2005 of like,
okay, you got your cash back and you got your coupon.
Great.
Well, what the heck?
What else is there?
There should be a heck of a lot more.
Your top 10 should have influencer, great, huge content publication network,
maybe some like really niche provider.
If you're in the baby space, you should have an insanely awesome baby blogger
or network that's promoting you.
That really nails your niche.
There's all kinds of tech that's blown up in terms of like FinTech and card
link offers where people are getting relevant suggestions when they go into
things like Acorns and NerdWallet that are relevant to a lot of people.
There's search providers that are willing to collaborate with you and hey, they
may not be in your top 10, but they're a consideration to look at to diversify
and push your competitors down.
There's email providers.
There's ways to tap media buyers that are sitting on the sidelines that are
phenomenal at Google, Meta, TikTok that you can run on a pure CPA and get
quality and not have to worry about fraud.
All of them need management, all of them need handholding, all of them need
rules and guidelines.
But if you don't look into that top 10 and see a nice diversified portfolio,
you're kind of missing the boat.
Got it.
So you sort of, what you are seeing is that there's probably too many brands
think about affiliate as only being like the coupon code sites and not enough
are thinking about the more the top of funnel, more awareness building ones.
And those are the parts that are maybe even more incremental, but potentially
more difficult to measure.
Yeah, I think it's a combination of difficult to measure.
Some of them do require some level of flat fee because they can command that
for quality and for size.
And so you want to, I think the common mishap is that it, they try to apply a
paid search or paid social methodology to a channel that doesn't operate that
way.
And it just needs a lot more cultivation and handholding essentially.
It's just, it's simply not programmatic yet.
And I think people think that they try to apply programmatic principles to
affiliate and that doesn't work.
It needs to be actively managed, really actively managed.
And so let's come back to like experimentation and how we might be able to do
this.
So like if I'm a brand and I want to run an experiment with these partners, is
it literally just like picking up the phone and being like, Hey partner, we
want to run this experiment.
Like let's figure out how to go make it happen.
Or what are the ways that you've seen this work tactically that a marketer
today could potentially go and run with?
Yeah, tactically and tactfully, right?
It's like, it's a sensitive topic, you know, when a big brand reaches out to
honey and it's like, Hey, we want to roll a holdout test.
Like, I don't, I don't think honey's like jumping up and down saying, can't
wait.
I think there's a number of things that, you know, if you have the tracking
dialed in really well, if you have no server to servers, a better methodology
than pixel, if you have the attribution dialed in really well, Hey, go beyond
last click.
If you have your pricing dialed in really well, like being really smart about
pricing, what you see on the data is higher quality and higher revenue and
higher unified customer.
You're already ahead of a lot of the players.
And I think until you get to a certain size of volume, so you can run an
appropriate test, you know it better than anyone in MMM.
You've got to have that sample size.
You've got to have that data set until you get there probably doesn't make
sense to approach a loyalty partner and say, Hey, I want to run a test, a geo
or a holdout.
Now, good news is for those that are more sophisticated, have gone through
those steps and have kind of gotten to like PhD level affiliate marketing, you
know, then it's time to approach the partners that you have questions about and
say, Hey, we want to run an experiment here.
Would you be open to it?
Here's how we're kind of thinking about it.
We'd like to hear your feedback.
And it's a relationship.
It's a two way street.
It's a negotiation.
It comes in ebbs and flows.
I'll be honest.
Some partners in that space are like, we can get this revenue elsewhere.
Good day, sir.
And they move on.
And I think that that's really interesting.
I think that the better partners are going to be the ones that are, you know,
in the right use cases are willing to make testing a little bit easier and
stand behind their offering with confidence and allow the brand to make the
determination based on it, assuming that the methodology is right.
I think where people get into trouble is when they throw the baby out with the
bathwater and say, Oh, coupon and loyalty don't work.
It's just a nonstarter for me.
I think it's ultimately more about how you value that partner than throwing it
out completely.
Are there examples where you need to remove and move on?
Absolutely.
But I think it's all more about the right, accurate tracking of value, which,
which is obviously what you guys are trying to do with what you're building.
Yeah, I think that's right.
And I think it's just it's such a it's such a tricky and hard problem.
And I mean, at Recast, we spend a lot of time on affiliate because affiliate is
really problematic, actually, for doing marketing mix modeling.
And the reason why is that in general with affiliate programs, you pay for the
marketing activity after the conversion has already happened, which is
different from how all other marketing works, where you spend money and you get
impressions and then conversions happen later.
In affiliate, it's reversed where you get the revenue in the door, and then the
spend happens for the affiliate partner.
And if you think about the way that like all of MMM is structured to work, it
sort of is implicitly making the assumption that the spend is happening and
then conversions are happening later, not the opposite.
And so MMM modeling is really, really hard to do correctly with affiliates.
And in fact, when we first started building Recast, what we found is that the
model was like too smart, it was too good.
When we would include affiliate activity in the model, the model basically just
found out what is the payment for affiliate conversions.
And we were the model basically just identified, hey, look, you're paying $5
for every conversion.
And so that actually doesn't help because then it's not truly measuring
incrementality.
It's just finding the relationship in the data, which is that when your spend
goes up by X amount, your revenue goes up by some 20X that amount, because
there's a 5X return on investment on that affiliate spend.
So it's tricky.
It's hard.
It's a really hard problem because of that closed loop system.
And so we have spent a bunch of time at Recast really thinking about how do we
break that connection and make it so that the model isn't just going to find,
hey, what's the affiliate relationship here, but actually thinking about how do
we understand the relationship between other marketing channels, the
interactions with affiliate, and then what affiliate is going to happen, like
what affiliate spend is going to happen no matter what, how much of that is
incremental.
But it's a very, very, very hard problem.
And so a lot of MMM researchers will just not include affiliate at all because
they're like, it's too hard.
It messes up the model.
It's really a tricky problem.
And I think until we as an industry figure out better ways to get really smart
about measuring affiliate, and I'd love to see people developing more tools for
doing this sort of holdout testing in affiliate channels.
I think it's just really tough for marketers to get a good solid measurement
that they have a lot of confidence in.
So as you said, they're sort of stuck piecing the different pieces together
from the different evidence that they have.
And so, I don't know, I'm really excited about the future.
But what I want to see is I want to see more of these like experimentation type
tools being developed in the industry.
So that way, if I'm a brand and I want to experiment with honey, I can easily
run an experiment.
I don't just have to take their word for it.
I can actually say like, okay, look, there's a holdout test.
We're going to have honey in half the country and not in the other half and
really see what happens.
Yeah, I think it's possible.
I think it needs to be more embraced.
I 100% agree with you.
An interesting thought came into my head.
I'm not sure if it would work, but there is a percentage of affiliate that does
do some of that upfront payment.
There is a percentage of affiliate that is, which is kind of debatable whether
that's affiliate or not.
But it's there and it's happening.
And so some will move to cost per click models, maybe to avoid certain legal
issues or constraints.
So let's say an MMM candidate brand working with Recast were to say, hey, look,
we're going to run a certain amount of our, like influencers is a good example.
A larger percent of that is like, hey, payment upfront for a multi-staged
campaign, let's say multiple posts, et cetera.
What if maybe starting with a slice of affiliate, like an, like Instagram
influencers for a retailer, if we were to say, okay, we're going to, how much
data would you think you would need for a type of brand to run?
Like maybe looking back over the course of a year, I'm just kind of teeing up a
case.
Do you think you could learn about the incrementality of that influencer
strategy?
Totally.
If there's enough spent, right.
And this is the thing that we've done.
And again, this is a thing basically like when we're working with these brands
and they have, they might have influencers, some of which are sort of
traditional affiliate where you're paying per conversion and some of which are
pay upfront.
And we'll basically split those in two and we'll treat them differently in the
statistical model because you have to, because of the problems I was just
talking about.
And so with that spend, right, the paid upfront influencers, we can treat that
like any other marketing channel because you're paying money, you're getting
impressions and then conversions are happening.
And if you've been running an affiliate program for a year and it's a
substantial amount of spend, yes, we can absolutely identify the incrementality
of that.
And we've done that with a bunch of our partners in terms of being able to say,
look, when you are investing more into these influencers, you're driving in
number of additional dollars of revenue or in number of additional conversions.
And therefore we can back into what the incremental return on investment or
cost per acquisition is from that investment that you're doing.
And that's actually like one of our key selling points at Recast is that we can
give you insight into channels like that, that otherwise are very difficult to
measure because as we all know, it's difficult to parse apart.
Like if you're running influencers on YouTube, right?
Is it your YouTube ads that are happening?
Is it the influencer?
A lot of times those people aren't clicking on any link or actually engaging
with that post in any way that would lead them directly to your website, but
they are, you know, maybe they're watching YouTube on their TV and then they're
going on their phone and searching for your brand.
You really need to have a good way of actually being able to understand that
connection, even if they're not necessarily using the coupon code or the vanity
URL, which I personally almost never use.
And so like I can empathize with those buyers that don't necessarily do that.
And so you need to understand what are the sort of statistical relationships in
the data in order to measure a channel like that effectively.
How much does the payment upfront improve your fidelity in your opinion in that
factor?
Or is it like number three on a series of factors?
So it definitely helps from an econometric modeling perspective.
It definitely helps, right?
Because you don't have that problem of the sort of circular causality thing
where you're paying after the fact of conversion.
And so from a causal modeling perspective or a causal inference perspective,
it's definitely a lot easier to get a read in that situation.
For the ones that are done just on a paper performance basis, we definitely
have a lot more sort of asterisks in terms of thinking about, hey, is this
measure actually as accurate as we want?
And then a lot of times that's the point where we're talking to these brands
and saying like, look, we should figure out how can we go and test this channel
to make sure that we're getting a good read here or get outside information
about the true incrementality so we can use it for real decision making.
Just to play out the scenario, not to like over dwell on it, but like if it
was, let's say a brand was spending 500 grand a month on affiliate as a
channel, let's say you had a 12 month look back, assuming a relative 50 cent
cost per click, we can do the quick math, right?
We can just model it out today on the call.
I'm just kidding.
Do you think you can get a read?
I mean, again, it depends on what else they're doing, right?
But that's a fair amount of spend.
You should definitely be able to start to model out what's the relationship
there.
Yeah.
You got to be making a lot of money through that lever to be spending that
much, which is probably not a lot of retail, not a lot of DTC income, but
there's some.
You had a 12 month look back.
The question is how much of that would you need to be paying on a cost per
click or the opposite of an affiliate model?
So it's like, does it justify switching over to a cost per click or upfront
payment?
I would encourage brands to experiment, right?
Like, can we carve out a small part of that and switch it over and see what
happens, right?
Let's take a test and learn.
I mean, this is our view on like literally every problem, which is like, let's
run an experiment.
Let's take a test and learn approach.
Let's figure out how can we test this and validate these assumptions without,
and we don't necessarily need to be like, let's make a whole big change for
this whole program all at once.
One of the small things that we can do to start getting learnings and that
could inform what the next step, we don't have to plan out the next two years
of our marketing spend.
We can say, let's run an experiment over the next two months and then take that
and use that to make a decision about what we do three or four months from now.
I love that.
It's totally in line with our philosophy and always be testing and
experimentation.
I think that there's so much alignment there and taking it like one partner at
a time, one geo at a time, breaking down the problem, the first principles.
I think that's really something that is there for people to try and be willing
to test and learn from.