Subscribe
Share
Share
Embed
Always Be Testing explores the experiments, insights, and growth stories shaping the future of affiliate and partner marketing in B2B SaaS. Hosted by industry veterans, the show dives deep into real-world lessons from the people driving measurable impact at companies like Google, HubSpot, Ramp, Webflow, G2, and beyond.
Each episode uncovers what happens when today's most innovative marketers challenge assumptions, run smarter experiments, and build programs that scale revenue through meaningful partnerships. If you’re in affiliate, partnerships, or SaaS growth — this is your front-row seat to how the best do it (and what they’ve learned along the way
Hello, hello, hello, welcome to another
episode of the Always Be Testing Podcast.
I'm your host, Ty DeGrange, and I'm here
in beautiful Santa Barbara at CJU.
We're talking to Catherine Tomargo right
now,
the AI Zar. How you doing?
That's
a new one. I love it. Doing great.
Can't complain really.
Just got out of a fun agency CJ
conversation,
diving deep into all the things and CJU is
near and dear to my heart because
I was a CJU employee and it's great to be
back in Santa Barbara.
We've got gauchos on our on our team.
I'm a former gaucho, so it's good to be
back.
And thanks
for joining us.
Yeah, thanks for having me. Yeah.
know a little
bit about what's happening all this AI
stuff so hopefully you can help
us and you also know a lot about CJ
because you've kind of been with CJ a
while.
How did you
Yeah.
how did you how'd you do that?
How did I do that? It's a great question.
I mean for many I'd say about
a decade now, we've been talking about how
affiliate is the channel of channels,
how it we can pretty much do almost
everything on a performance basis.
And I think I just got really lucky early
on with clients that were,
you know
~ so I was able to to test into
a lot of new emerging models and that
eventually just sort of,
you know, set me up for for being at the
forefront of what's happening now within
Yeah, it's come up so much on the pod,
it's obviously the namesake, but you you
lived it and made sure that
it was happening for your clients.
It's something that Tom and other
colleagues and I have talked about,
which is beyond important. Coming up
through CJ,
was there you know was there something
that sort of stood out for you
or or taught you with those experiments
that kind of like how did you get to be
like
Obviously it's become massively important,
but I'm excited to hear how you went from
CJ Client Services to leading
the AI charge procedure, which is really
really amazing.
Yeah, so I mean again it it comes back to
this test and learn framework.
That's something I really encouraged a lot
of my clients to ensure they were always
doing and with testing and learning
they're you know,
you gotta look at the data. ~ so just
having that
Data focus or data focus just persists
throughout everything that that I do.
Earlier, I would say like late twenties
four,
early twenty.
that I helped develop from CJ around
understanding where referral traffic
was coming. ~ where for both, you know,
publishers and where advertisers were
referring traffic,
et cetera, et cetera, ~ was able to pick
up on the fact that
early days, maybe not so much right now,
but early days, ~ some of these emerging
platforms were actually driving lots
of traffic to publisher sites.
And so quickly understood that as both an
opportunity ~ to
improve that for publishers, right?
So if if you're not yet receiving kind of
traffic,
how can you do that? And because there
wasn't a ton out there,
it's really required just for the digging
in and we are.
Yeah, absolutely. And when clients are
coming to you and coming to CJ to say,
I need help figuring out AI visibility,
what are some of the things that they're
getting and how what does that
look like?
Sure. So we back to data,
right? ~ we've spent spent a lot of last
year and beginning of this year just
trying to lay the foundation,
right? Or or understand and unpack what it
is that we
the tech stack should be ~ since the
existing tech stack doesn't quite tackle
a lot of the pieces and so ~ with that
research we've developed this sort
of we'll call it our own tech stack or or
suite of tools ~ that allow brands
to understand that starting with
visibility right so like what visibility
far as we're concerned and I think most of
the industry is whether or
not a brand appears across these
environments and and how they appear.
So there are tools in place that allow
brands to track that.
That's sort of where we start ~ is
establishing those baselines
or conducting an audit in order to
establish those baselines.
And then from there looking at the sources
that are driving that visibility
on behalf of that brand to that then you
know we can then devise a plan.
But it's all grounded in data.
I
love that. And what you shared earlier
today was really cool and like citation,
sentiment, mention, like you talked a
little bit about like what that maybe
looks
like without giving up the whole story.
It's a really I loved what some of that
visibility you shared.
Yeah, sure. So with this this new touch
point we'll call it,
~ because it hasn't yet received the full
~ channel crown just yet or designation.
but was just information
Okay.
here on the pod. Not quite. I just think
the industry hasn't fully aligned.
Like you, GEO, are a channel now.
but the
It is new and it is different.
~ it is similar to
Other things that we've done in the past,
but it is nonetheless different.
And where it sits is really like in this
upper funnel space.
Yeah. And as such the the KPIs are
different.
Right. So we talked about visibility.
Yeah. That's one of the main KPIs.
How is your brand appearing? That's what
that is.
On a per kind
Problem.
of basis. Yeah. A prompt, a platform,
etcetera. ~ so that's a visibility score,
which is also known as pension rate.
Those two are somewhat interchangeable.
~
And then there is ~ sentiment,
right? So like you're appearing,
but is it positive or negative?
What what's the deal? ~ and then position,
right? So you're being listed,
that's fantastic. Are you first or are you
last?
and those are the sort of four KPIs that
that we look at when we think about just
Yeah, that's amazing. A lot of talk
happens around path to purchase,
attribution, MTA, all that stuff.
There's like kind of the brand led
conversations,
AI led, social. You know, where where does
C A CJ see it when with AI growing?
What's like your is there pieces that you
think are kind of more overhyped?
And what where do you kind of stand on
that those paths
to purchase?
Yeah, I mean look data exists that shows
us that AI referred traffic is converting
at higher rates than
non-AI referred traffic. So that's not
that's not made up or overhyped,
that's that's the truth. The the rate at
which it's higher does vary depending
on the time frame or platform,
etc. But that's true.
Tack that
call them answers, right? We used to rely
on search engines to conduct
our own research and
Mm.
~ that's another thing that that's that's
not overhyped,
that's that's just the truth, right?
Even if you're not an AI adopter,
you use Google and Google has AI
overviews.
So ~ it's pretty much ~ reached everyone.
So some of the things that might be
overhyped or around some of the
guess measurement that can be delivered at
this point in time.
But as far as whether or not users are
using it,
if that's not overhyped. Absolutely.
as far as whether or not brands should be
testing,
I I don't think that that's overhyped
because that's happening now
and it's better to be in a position of
being proactive versus reactive.
And there's a lot to learn right now.
~ a lot of the conversations I have with
with brands are around
though can't quite forecast just yet,
even though we can't quite project the
returns,
whether it be, you know, anything further
down the funnel.
Sales or sales revenue. A learning is a
win.
and that's something that you would want
to know.
Like how how does being visible translate
to those downstream ~ APIs?
So I'm not I think that the framing of
some things are overhyped,
but like the fact that you should be doing
it and that it is important is not.
Yeah,
and that's a perfect segue into,
you know, namesake of the pod,
the testing importance. We've clearly
established that like,
hey, you kinda have to be doing these
types of tests.
What's a way you like to see it structured
in a GEO,
answer engine environment? How would you
like to see a test structure appropriately
and set up for success?
So
~ it all goes back to the data.
Yep. ~ we the data's not absolutely
perfect and what I mean by that is
in an ideal world we're getting exact
query or or prompt information from
the PLMs themselves alongside the exact
responses and the sources cited.
In the absence of that we have synthetic
data that we can look at,
we have CDN call signals that we can
incorporate.
a number of things that we can we can look
at to then help help inform what that
test should look like.
Can
you say some credit data, can you share
what that means exactly?
Sure,
so ~ in the absence of receiving the
actual prompt information from platforms,
there have been a number of emerging tools
that have emerged over
the past couple of months, eighteen months
or so,
that run synthetic prompts across those
top LLMs,
capture those responses, and that's what
they then use in order to put together
scores tell you which sources are cited
and how frequently across which topics,
etc etc.
Amazing,
amazing. Are there publisher types that
are starting to really nail it from
a GEO citation perspective? What are are
there are there types of reviews
or types of formats that are really
winning in this Answer Engine environment?
Sure, I mean honestly it's the ones that
type closest to the types
of prompts that consumers ask that are
very structured in a very clear
and succinct way. ~ things like the best X
of you know for whatever
gifts to buy mom, dad, brother,
son, etc etc. ~ these are the kinds of
this is the kind of language that's easily
digested.
Yeah. As far as types of publishers,
it really comes down
industry
Large percentage of affiliate publishers
were being cited or were being cited.
Each individual publisher has taken a
different approach in terms of
how they want to perceive those citations.
Some really appreciate that and I
Optimizing their schema and their
structures and the type of content that
they focus
on. Yeah. And others are
So it's not so much about the type of
publisher,
although I'll tell you can't you know,
publishers that write things on the
internet,
which is everyone, ~ are the types that
could be cited.
It's those that are are really leaning
into it.
I love it. I love it. And we talk a lot
about zero click.
We talk a lot about last click being
flawed.
We talk a lot about the need for more
visibility and exposure.
And we're working on that ourselves.
I mean we we have methodologies to
showcase clients visibility.
We ha we talk and often talk about how
it's a hey,
it's a gift with purchase. You're getting
core partner marketing,
but a massive percentage of this is
showing up in AI citations and GEO.
I've I've talked about this a little bit
and I've written some things about payment
and how do you tie payment and
compensation to this world?
How does CJ I mean how does CJ think about
it?
How do you personally think about it?
~ the compensation models, how do they
evolve in more of
a visibility environment when the click's
being lost?
Sure, so
I mean my personal POV is they absolutely
need to evolve,
right? as search,
I'll just say search, refer traffic
declines for publishers,
they they need different ways to monetize
their influence,
right? ~ and so there are things that we
can use in
the absence of those things that we need,
right? So we talked about those things.
talked about ~ CDN call signals there
there are a lot of things that we can pull
in.
our univers well
This we might need to cut.
CD and call signals, other technologies
you can share
all digital data so that we can analyze
trends,
you know, direct conversion trends and
apply ~ or create methodologies that then
allow us to to make assumptions around
publisher influence ~ to conversions.
And that's something that absolutely needs
and we need to begin testing. I think in
the intern we're focusing
on flat feed engagements as we develop
those,
paper influence or just ~ dynamic
commissioning structures based on that
engagement.
No, it's good. I mean I think that what
I'm hearing is you're thinking about
a lot of ways to compensate. You've got a
number of levers,
you know, kind of within the existing
ecosystem to compensate.
And then there's all these new ways to
measure that we can co tie
to those payment methods that allow you to
have some flexibility around it.
So I think it makes sense too.
But it is an industry priority ~ that we
all need to start thinking about.
~ particularly I would say on the pub
publisher front,
right? ~ we I work really closely with
with publishers,
wanting to hear from them, wanting to hear
that feedback to better understand what
it is that they need. Yeah. Because at the
end of the day,
they're the ones providing the content,
they're the ones driving that influence.
Yeah.
~ and so we wanna make sure that whatever
whatever is developed is
Yeah, absolutely. I mean we often talk
about it like they're the
RD of the ecosystem, they're the currency
of the ecosystem,
they're they're really who is driving a
lot of it in so many ways.
So I love that framing. That's great.
It's mythbuster time. We ready to bust
some myths?
Yeah,
they're what's some
Let's do it.
good what's some good like Geo myths that
we want to just kind of say ~
Not really not really your jam.
meh.
Taking performance budgets and defining
and leveraging those for upper funnel.
If you do that, you're gonna kill
performance.
Like if you remove performance budgets,
then your pr you won't meet your
performance goals.
those th that maybe it's not a myth,
but we'll call it
Is it a mistake or trap that you see
brands saying we're going
to take away
Let's
reallocate performance budgets for the
purposes of GEO,
which then impacts the performance.
Yeah, yeah, you need you need the fuel of
what's working at the very least.
And that should that should be coming from
another experimentation budget or yeah,
you're gonna lose something. One of many
silly claims I
Yeah. The the
Yeah.
Another big myth. GEO is primarily owned.
Or rent it. Yeah, yeah. Yeah.
~ the data shows that the vast majority of
sources cited are third party sources.
Go through
Yeah. If you want GEO
results, you heard it here. Kind of
important.
Kind little bit.
~ big big myth, there are clients we talk
to that I'm like,
all right, let me know. ~ would love to
hear more about what you're doing.
They're like, we got this down.
We are re like ~ rewriting our entire
website.
We are writing so much content.
We are doing all of these things.
I mean you kinda need both, right?
But you don't wanna neglect definitely
don't wanna neglect th third party trusted
content that talks about you in a way that
doesn't feel as biased
as you talking about.
Yeah, but it's not even about the bias,
really, it's about
Let's look at the
Like it just is let's go back to the data.
And that's really what ~ I encourage every
company to do.
let's start with the data and who is being
cited when it comes to your brand.
That's that's then the work that needs to
be done.
And I'm telling you, ~ their analys I
There are analysis that I have read about
on the internet,
~ including analysis that CJ has done and
published,
that shows that over 83% of sources cited
across verticals are
Yeah, that percentage sounds like it keeps
going up.
I kinda like that. good myth busting.
The czar of AI, bespoke. Agentic.
What the heck is it and what's going on
and what's coming?
A
gente. So you know,
I sometimes I worry about hot takes in
AI because they're always like famous last
words.
Yeah. a gen tick in particular,
that one feels it feels a ways away.
Yeah. I sure I'll tell you something
anecdotally.
Yep. Last year.
Last year my friend and I were were
talking about back to school.
She was doing back to school shopping for
her daughters and we built some agents.
We're just like, ~ my god, AI is just
going to simplify all of our lives.
Let's get some agents to get that done and
boom.
About the
wrong stuff.
I listened
to a keynote today ~ that talked about how
either half or over half something like
that of the internet is ~ it's actually
bots.
It's bot traffic. So you know what a lot
of websites have done?
They've implemented ways to block bot
traffic.
And so work agents. Until thank you.
~ so until we really like
solve for that that piece and of course
there's like authentication and and
a lot of but there's a lot of work to be
done
Turns out we're still employed.
it what a
concept but even beyond that I mean
there's there are two different maybe
three
but the two main ~ challenges that we need
to solve for is one consumer trust right
like who I I don't really know that many
people that are handing over
are ready anyway to hand over their credit
card to agents and have them
~ fully make those decisions. And then
there's the the website or internet trust.
The websites don't trust the agents to
come in and and make purchase decisions.
So until those two things are solved for
I'm just I th I think it's
I think it's always
right. Yep, Katarine, head of AI at CJ.
Miss 305, some people say. It's
That one says.
a it's a it's
it's a it's a starting to get traction
here in Santa Barbara.
What's one big takeaway you want to give
the audience in the AI realm that
you recommend they they take action on?
Yeah.
I would say
start collaborating cross channel,
cross cross almost everything really.
~ as we dig into as we dig into
how these LLMs work and and how they
formulate answers,
we're really finding that every single
channel plays role.
And when you are having one channel focus
on all of it,
you're really missing a lot of context and
a lot of
the opportunity. So bring it all together,
create a center of excellence,
create a tiger team, create a strike team,
whatever you want to call it. bring in all
the right stakeholders
and focus on the data.
For the love of God, please look at the
data and make decisions based on on what
it
I love it. Sounds obvious, but it's
clearly helped you navigate all all
the way up through your career with one
great company and you've been able
to lead it now with AI and it's been a
pleasure to get to know you here
and thanks for joining us in a pretty nice
location to to figure it all out.
So until next time.
Thank you so much for having me.
Yeah.
Thanks everybody. I