Subscribe
Share
Share
Embed
We share our thoughts and ideas on how to grow a business.
Gemini three has an explicit flag
in their system where if a brand
is being mentioned in a commercial
prompt, it does not cite that website.
Wait, wait, wait, wait, wait.
Hang on.
You're saying if you mention
yourself in the post.
You can't both be cited and
mentioned in the overview.
Look, look at this.
They are using like complete
and utter like crap.
That has nothing to do with what you
would normally respect as SEO content.
1.0 was about targeting keywords
and the intent of around a keyword.
And then content in the 2.0
is going to be about figuring
out how to write content that.
Best matches to that persona.
And so how do we influence
that personalized search?
Yeah.
My understanding from the OpenAI
perspective is that it's black
box seat that OpenAI doesn't
reveal what goes in the training
data, how often they update it.
Is that true?
I think Gemini is just gonna win.
I think GPT is going to like
go to like obscure irrelevance.
Why?
It's because they get search.
When you search as a model,
your answers are like anywhere
between 10 to like 200% better.
So what we found after
running, you know, like.
Tens of thousands of, of prompts was that
Gemini three has an explicit flag in their
system where if a brand is being mentioned
in a commercial prompt, it does not cite.
That, that website.
So if you look at like best SEO tools,
like yes, you might occasionally get
hres like blog, like own blog posts
like mentioned, because Google does
still perform the search and it would
be like captured in that like mechanism.
But they're explicitly ignoring
basically like if we were to write
best, you know, SEO tools at Clearscope.
Then we cannot get a citation because
Clearscope's being mentioned, uh,
as like one of the top like tools
within SEO, which then kind of defeats
the purpose of Wait, wait, content.
Wait, you're saying if you mention
yourself in the post, you can't both
be cited and mentioned in the overview.
I am saying if Google is citing, or
if Google is mentioning your brand
like SEMrush, HF surf or Clearscope
phrase, whatever, then you cannot
qualify to be a source for that.
Like, uh, for that.
Like, oh, I see.
So you're saying here best SEO tools,
the a IO review mentions SEMrush.
Arus, S-E-O-S-E.
And then if you look on the
right, you have US chamber.
So they'll never be a source here.
It's not never, but it's very
dramatically suppressed currently.
That's interesting.
'cause when we look at like best content
marketing agencies we have, we were
like for SaaS, we're like one of the
top sources and were mentioned really.
So maybe it's just an A,
like a software thing.
Yeah.
Maybe.
Unless we're not mentioned today.
Oh, there we're.
Yeah.
Are you mentioned or are you cited?
Oh, you are?
Yeah.
Okay.
Yeah.
We're actually seeing that
a decent amount actually.
Interesting.
I, I, I think it's a function of
how you write a lot of that content.
Hmm.
Yeah.
Like there, there's the lily ray,
like blog posts and tweets about
self-promotional listicles are like
hurting your SEO or hurting your
ability to, to show up for that kind
of stuff, or they're just not ranking.
We're not seeing that at
all across our clients.
Like we have a number of these.
Listicle type blog posts where the,
the client is both ranking or like,
has a citation and being shown
as one of the recommended brands.
And I think it's just a function
of how the, the post is written.
Like I, I think a lot of the,
the posts that maybe she was
talking about were like, clearly
ai, clearly self-promotional, not
identifying themselves as the source
of information, like in, in the,
the way that we write all of 'em.
We're explicit in saying, here's why
we're putting ourselves as number one, or
Here's why we're ranking things this way.
And I don't know, maybe that helps.
But yeah, we have like a number of
different clients where like one
of, one of our content pieces is a
source of information and it's also
being pulled as a brand mention.
Yeah.
I'm not saying it's impossible,
but the other, like, here I'll just
show you the other really deep like
experiment that we were diving into.
Like you guys know, like temperature and
like all that stuff and the, the whole
like Google, like ecosystem, right?
Like temperature and, and stuff.
Like it defaults to one.
Alright, so I, temperature in AI
world just refers to how much.
Creative freedom you give to the model.
Okay, so essentially, as you can see
here, when I hover over it, um, Gemini
three best results at default 1.0.
So 1.0 is basically what most
models like by default use
GPT Claw, so on and so forth.
One gives them a good amount of freedom,
but it still provides guardrails that.
You know, two, like two is just like,
you know, make stuff, make stuff up.
But zero zero is, you cannot deviate
too much from, from the training data.
And so essentially then we say, okay, well
zero is basically Oh, your way to figure
out what the training data defaults to.
Aha.
Yes.
So if I do this best content marketing
agencies, right, and I run this with
Google, like grounding with Google
search, what you'll see is that you get
recommended, but you don't show up as a.
Citation or source it.
And this is across the board, right?
This is best SEO tools, right?
Samr A trusts, SC ranking, screaming
Frog surfer, clear, skip, whatever.
But look, look at this.
They are using like complete
and utter like crap.
That has nothing to do with
what you would normally, you
know, respect as like an SEO.
Same thing happens here.
Well, I guess 10 speed.
I don't know if 10
speed actually shows up.
Here.
Nice.
We're in there.
Are we the complete and utter crap?
No, no.
We're actually mentioned.
Yeah, yeah, yeah.
You're, you're here.
But you can see here, right?
Like basically nothing.
Nothing.
Yeah.
Those are interesting
sites to be pulling from.
Yeah.
They're like oftentimes
actually like absurdly bad.
Well, I don't know about the agency one,
but when you look at like this, this
crap, like you've gone in and evaluated
the content itself and it just looks bad.
Yeah.
It's like, what is this?
Oh, that's, this is nothing.
That's, yeah, that's actually, yeah.
And this is, this is training data.
This is as close to the training
data as you're gonna get.
Well, the output is training
data, but it's still grounding
in that, so it's mixing.
Yeah, yeah.
Yeah.
So basically what ends up
happening is that it's not
run at zero, it's run at one.
And so when it's run at one, there's a bit
more creative freedom, like, uh, freedom.
But I, I don't have the direct,
basically we found that like a
trusts ended up getting cited
like 40% of the time because.
Google searched and after they searched,
they like determined that like that
piece of content was like good enough.
And then they used that like as a source.
But by default the the answers are being
constructed without those like criteria.
And then Google will search at
temperature one, and then from there
you then get to possibly vie for it.
But there's definitely a suppression on.
What is happening in terms of brand
mention to source, and I think this
is what Google's way to say, like,
okay, if I mention you, I shouldn't
look at the content you're producing
because it's going to be biased.
And that's like why self-promotional
stuff has basically fallen off a cliff.
Obviously if you still do it
tastefully and do it well, you're
still gonna rank in Google search
and then you're still going to.
You like potentially win a citation or
a source, but by default, like that's
been erased from the current like system.
Yeah.
Again, like, but we're not seeing
that like we do that for our clients
and it's, and it's still working.
I mean, we, I have to look carefully,
well, here's like, a theory that I have
is that when, when you're doing marking
and really crowded or competitive
categories, that it is much harder
to influence the search than it is
if there's not a lot of information.
On that topic area.
So like the example that I have is we,
we started working with this trucking
management software and before we started
working with 'em, there's just not a
lot of good content in that category
because there's not a lot of software
options, there's not a lot of information
on like what this category does.
And so all of the content that
we're producing for them is pretty
novel for that entire category,
and it's helpful and educational.
And they're getting cited like crazy
and, and they're also showing up as the
answer just because there's just not a
lot of good information on that topic.
But then if you go into a space like
marketing, like where, where you're
trying to influence content marketing
agencies or SEO tools, I, I think
what's happening is they're taking the
brands much more like the top brands
much more into consideration mm-hmm.
Than certain list posts.
Or like certain pieces of content.
So like they already have a sense
of, for SEO tools, what are, what
are some of those top brands?
And so it's much harder to influence
a search for like best SEO tools
or best like, uh, exercise bikes.
Like Peloton is just gonna be that known
brand and come up over and over again.
And some new incumbent in the space.
Unless you get a ton of like people
talking about you on multiple blogs
across social, then that could
potentially change like the outputs.
But I think it, it's much harder
to influence for those categories
that are really established.
I don't know what you think about that.
Yeah, yeah.
It is, it's probably something
there, um, like as in, because
there's no content there.
Like the, the model is essentially
like forced into just accepting that,
um, content being produced is, is
like they have to use it essentially.
I, I mean, I can see that, but I also
think for like a lot of the searches
we're seeing, the searching the web take
place, it's very rare unless it's like.
More of a top of funnel query or
mid funnel query that you wouldn't
see that search take place.
And so I think there's two things
happening on top of the grounding data.
There's the search happening in the
background to try to find more context
or just like check its answers.
And then there's also personalization
happening, and I think the personalization
happening is probably where a
lot of the opportunity lies for.
For SEO or GEO, because if we just
think to the future of search, like
Google was based around keywords.
Now with LLMs you have much
more identifying information.
And if the goal of the LLM is to
try to provide the best answer
to the user, it's going to use as
much data about that person as it
has to provide you a great answer.
And so I think the
opportunity lies in content.
In the SEO world or like con content,
1.0 was about targeting keywords and the
intent of around a keyword and targeting.
And then content in the 2.0 is going
to be about figuring out how to write
content that best matches to that persona.
And so how, how do we influence
that personalized search?
And I think.
The answer comes to less around these high
level topics and more around specifics.
And so for example, if grow and converts
best clients are SaaS companies, the
more that we can show that our content
marketing strategy works for SaaS
companies in terms of the content that
we're writing, the case studies that
we're producing, that map to those
personas, like basically every piece
of content we make as specific as
possible provide real world examples.
Talk about the personas, who would use it.
I think that's gonna influence the
search a lot more down the road
because if someone's saying, Hey,
I'm in a mid-market SaaS company and
this is my role, I'm looking for an
agency that can help me grow leads.
It's saying, huh, grow and convert
positioning is all around driving lead
gen through content, and they have a
bunch of case studies on their site.
That are also talking about
serving that audience.
I'm gonna recommend this company
because it best matches that.
Like persona of the person searching
I'm on board with with all of that.
I think it's maybe just like the
role of like informational content.
So what you're talking about is
what I've like kind of seen happen
is just like at the end of the day,
if Clear SEOuts gets cited for like
how to do SEO, like, I don't know,
what's the marginal value of that?
I mean, obviously I could like
hypothetically and philosophically
be like, okay, well if we get cited
then you know the models must.
Think, you know, Clearscopes is
topical authority in whatever,
internal linking or whatever.
And that's probably a good
thing, but you know, I'm not
really getting traffic from it.
Instead, you know, what I I see are, uh,
what, like we get cited for is like our
pricing page or you know, our feature
like page and you know, that maps to
like what, what you're talking about.
Where it's like, it's 'cause they
already know you as like an SEO tool.
So for example, if you run that search.
And they're citing your pricing page and
some like information about your product.
They're just trying to serve the
best answer because they wanna
pull your current pricing and
the current positioning of the
product to display in the answer.
Right.
But I, I, I think if we're taking it
then like a level further, I think
there's still opportunity to show up
for other bottom of the funnel searches
and influence those answers by being way
more specific in what you're targeting.
So if we think about like just
our framework around like.
Pain point SEO, I still think it's
valid, but I think the, the difference
is that the content that you used to
write used to be for like best SEO tools.
Whereas now I think you need to be best
SEO tools to do this specific task or to
like update content to produce new thing.
Like to actually have to be
tied to specific use cases
or around certain personas.
And I think that's kind of
the gap where I don't see.
Companies thinking about yet to where,
I think if you're way more specific in
writing those pieces, if again, if there's
not enough information about, okay, within
this SEO tool set, which are the best for
these specific use cases, then I think you
can influence those answers a lot more.
Because if people are running these
really specific searches, like we just
wrote a, a post that showed one of
our clients now when she first was.
Like in an LLM querying about what?
Content marketing agency.
It was like, it was like paragraphs of.
Information, like I'm, I'm doing
this, I've tried this before.
I'm looking for an agency
to do this specific task.
And then it recommended us as number
one because we had case studies that
closely tied to what she was looking for.
And so I, I think that's kind of
like what I'm thinking towards
the future is it's the specificity
that's really gonna matter.
Everyone has written this really high
level informational content, like the
how, like your example, the how to do.
SEO, but not many people have then gone
to like even more long tail stuff, and
I think the long tail stuff is going
to be where the opportunity lies for.
This next adoration of search.
Yeah, I'm, I'm a hundred
percent with you there.
So then let me just show you what, where
we're thinking about at Clearscope.
I mean, it is probably very relevant
to what y'all are, are doing.
I'm excited as well.
So, a couple of things, again, we
have, we've been doing a lot of
internal testing, just the whole
industry's fucking moving crazy Lee.
Um, so, um, all right.
I'll just start here with like the.
Uh, a lot of like the data
sampling that we've done.
This is primarily we were an
analyzing Gemini and chat, GPT,
obviously the top two models.
So, number one, what's interesting
is that basically of all of the
prompts that we run, Gemini searches
a hundred percent of the time and
ChatGPT only searches about 40.
This is somewhere between
50 to 70% of the time.
After more.
Data has been like analyzed.
So basically chat GPT only searches like
half of the time to 70% of the time.
And then when, this is the more
interesting part is that when
Gemini searches, it hella searches.
Whereas when chat GPT searches, it
usually just does a head term, like
sort of search is, is what we found.
What do, what do you mean?
So like it's breaking down that
search into like more like the
query fan out or like multiple
topics or It's searching for.
Like a wide variety improve.
Yeah.
This is what are the best
SEO tools sampled against
Gemini three a hundred times.
And then basically what we found is that
basically it's all, it's all probability.
Right?
That's prob, that's the whole idea is
the probabilistic nature of, um, LLMs.
But we found that within the probabilistic
outcome, it's actually very boxed in.
In, once you sample it like a hundred to
a thousand times, it normalizes around
it and it doesn't change that frequently.
It's basically the large language model
is constrained to create an answer
within a certain box of possibilities.
And when you sample it enough, you
get the long tail of possibilities
all mapped out by frequency.
Right?
So this is a hundred times.
Run.
What are the best SEO tools at temperature
one against Chad, GPT, and Gemini.
And you see here that 49% of the time
it searches best free SEO tools 2026.
27% of the time.
It's this, basically what you're referring
to is what we'd call the long, long tail.
Just the very fact that, you know, like.
Um, models Will, will do these
really crazy things, and these are
affected by user context, right?
It will be like, okay, you're
an enterprise or you know, you
want whatever category or you
care about local or whatever.
Anyways, these are the long, long tail
of what you would then want to basically.
Topic, cluster and then create, use
concrete use cases and stuff around,
because these are going to be, um, the,
the stuff that, that people care about.
Okay.
This is super interesting then.
So it does support this hypothesis
that yes, I'm completely a
hundred percent with you here, so.
Gemini searches a lot and it
searches all the time and chat.
TPT does not search a lot, and when it
searches, it searches one or two times.
This is confirmed over
here on, on the right.
Also, when I'm talking about
the citation thing, right.
I'm looking at like, again.
Got it.
We're sampling.
Got it.
Okay.
We're sampling a shit ton of stuff
and like when I look at like Ahrefs,
it doesn't fucking exist in a
hundred samplings of citations.
That is really odd.
Like this is.
Okay.
Seven per right.
Like it exists except it's, it should
exist way more like, but it doesn't, and
that, and that's what's leading to your
theory of saying you think it's being
suppressed, intentionally suppressing it.
Yes.
Because it's cited to sort of Yes.
Avoid a sense of bias or whatever.
Yes.
SEMrush frequency mentioned 100%.
samr.com suppressed.
Yeah.
I wonder if it's just like I,
yeah, I don't need to read that
because everyone has this, another
one is like another, this, this,
I don't have data backing this up.
This is just a theory of mine is that I,
I wonder if it just is like, I already
know about SEMrush Ahrefs from all of my
training data slash past searches, so I
don't need to be searching their site.
I already know this.
But the thing is, when you, when
you think about why it's searching.
The reasoning as to why the model
searches is because, A, it wants to
make sure that the answer, it's about
to give to the end user is factual.
It's up to date, it's
fresh, and it's relevant.
Yeah, in those cases it should absolutely
be looking at, yeah, like there was
a time where chat GPT would always do
site colon semrush.com, official site
pricing features, like whatever, right?
It's like I need to know exactly
what the site is saying because
I'm about to recommend it,
so it's actually disappeared.
Search format has like disappeared
from, from this like, um, analysis bit.
But, um, yeah.
So we saw that 10 are, are we, are we sure
that this is exactly how it works though?
There's nothing added on top of this,
because again, like if we, if we look
at, let's say, citation, citation
data and tracker, or just when we're
looking in a serp, we, we often
do see the, the company's content.
As one of the cited sources.
So I'm just cu I'm curious then.
I mean, you're clearly being suppressed.
We'll just put it like that.
Like I think there might just be
like, you know, more like, you know,
like why would digital Elevator.
Elevator, yeah.
Who that Series X marketing.
I know this person like Joe Robeson.
But why these be like above you?
They shouldn't except for the
fact that when we look at this.
You know, you are here or
they, you know, siege Media.
Like if when we look at
Siege Media like 24, right?
Like you're clearly being suppressed.
Like it's, there's some
sort of suppression that's
happening here, like, yeah.
Dave Daish, what's the, the, the
study that we just recently did,
what is the piece around the brand
side versus the prompt that we
were talking about the other day?
The one we haven't
published yet with Caitlin.
Yeah, because I, I wonder if that
kind of like gives some hints here.
That study is just chat pt and we
were looking at the overlap of what's
ranking for the two fan out queries.
It's like two to four that it shows
up in the console, what's ranking
there versus the citation list.
And I think we found like there's
a 25% of the things that are
ranking for the fan out query are
actually in the citation list.
So then the answer is like,
what about the other 75%?
The question becomes.
That means like the ones that are
ranking, only 25% of them are being cited.
Actually, maybe it's best
to say it the other way.
25% of the things that are cited
are ranking for the fan out queries.
So where it's, where's
it getting the other 75%?
Yeah, I mean I think that's, it's coming
from training data and training data
currently is very weird and opaque.
Yeah.
So we have the piece, it's like gonna
be published next week, in fact.
But, but wasn't there something about
like the brands that are showing though?
No, that's, that's a different study.
That's Rands and Rand just did, like,
he just asked, had people ask a bunch of
times, and there's this huge variance in
the brands that it actually recommends.
Yeah, well that's why we like
sampled it like a basically a
hundred times or tens of thousands
of times to like understand that.
And then what we found was that there
is variance, but over a large data set,
it's all normalized to probability.
Like it just, it all boils back
down to like, I mean this is all web
searches, but you know, you can, you
can imagine if we looked at it from a
brand perspective, it's all different.
Because that's the design, but
the frequencies over a large
period of time become normalized.
And then that's like what, what you,
I think that's what Rand concludes is
that, like at first he's like, look,
if you ask for the best dental CRM or
whatever, you know, uh, uh, there's like
87 different CRMs that are mentioned.
So like, what are we even doing here?
Right?
How, how do you optimize for this?
But then later in the piece he says,
but the top three are mentioned.
Often, like in every time you ask,
the top three are very likely to
be mentioned over and over again.
So if you become one of those
like well-known brands, then it's,
it's bound to mention you over.
Yeah.
Like you're, you're saying siege here
is, is mentioned every single time.
Yeah.
Almost every list, which,
which is exactly what we see.
So then, then the question becomes
how do you affect frequency?
Yeah.
How do you affect frequency is here.
Like, this is our guess.
This is our guess.
Right?
Agree.
It is your guess too.
This is the long, long tail.
It, it, it ma it actually makes sense
then because like, so the answer is
then you have, you have to produce
a bunch of really valuable content
that people are sharing and then also
that people are writing about you
as one of the top brands or like the
influential brands in your category.
Because if you couldn't affect that
frequency rate over time, then those
top brands would just always stay
there and there would never be.
A new product or a new service offering
that competes in that category.
Correct.
And, and so marketing has to be able to
affect the frequency that you're shown.
Yeah.
So anyways, lots of interesting stuff.
Um, no, this is super interesting.
So, because we've, we've been
diving into this too, but this is,
this is even more interesting than
the studies that we've been doing.
Yeah.
I mean, we're trying to figure
out what's the, the product and,
you know, obviously, um, you could
benefit from this information too.
So Gemini always searches.
When it searches, it searches a lot.
Chat.
GBT doesn't search that much.
Um.
Yeah, this is what, this
is very interesting.
So we ran this over time, right over
like seven days, and basically what
we found is that an initial sampling
of like basically a hundred, basically
from an informational query perspective
captured the a hundred percent of searches
that were run throughout the week.
So, you know, what does
SEO basically perform?
No additional searches
outside the control.
Over a week.
So it was basically a stable.
Right, like what is SEO is
always gonna be, what is SEO?
And therefore the range
of searches never changes.
A research based intent would be
like, you know how to do this.
A fresh intent would be like,
what are the, you know, latest
SEO trends or, you know, price of
Bitcoin or FOMC meeting or whatever.
And then a commercial
would be best SEO tools.
So then we saw, of course, fresh
should search more research searches
a little bit less in terms of
deviation from the control set.
And then commercial actually
doesn't search that much either.
So the, that, that graph on the
right, it means the gray means
it's running those queries.
The gray meant that within
the first hundred queries
that we've already captured.
All of those searches that was like
that, the model planned on running and
over the next seven days, you know it.
Came out with 17 to 23 in fresh
and research, but only, uh,
whatever, 17, 23 new, different,
new search, new web searches.
Yes, that's correct.
And a new web search would be something
that could be in the, like the 1%, right?
Like this.
And so, yeah, it's like even if it
got a new search, the frequency of
that search might be still really low.
So this brings you to then,
like, what about all these like.
GEO hacks around, like
posting on Reddit, the schema.
To me, none of this stuff
makes a lot of sense.
No, it doesn't.
I mean, you know how it goes.
There's always charlatans and
gurus and maybe for pockets of
time that works or whatever.
Right?
For sure.
But it, it's not a long-term strategy.
And like I, I find what's interesting
is like the, the metric of success
that everyone has around GEO right
now is, does it show up as a citation?
Okay, great.
But a citation doesn't
influence an answer.
Like those are two different things.
And so I feel like a lot of the case
studies that people are showing, they're
like, I produced this content and now it's
being brought in as a citation source.
Okay.
The citation source for what?
Prompt, first of all, because it
can be used as a citation, but
like what is, what is the actual
prompt that's being run that?
Yeah.
So like no one's really
talking about that.
And then the other thing is like,
like if we're looking at those sources
of information that is pulling from.
Like, I'm not seeing too much.
Social like, or like Reddit
or these review sites.
I can see maybe for that, like when you
ran best content marketing agencies.
The second thing was reviews.
So yeah, maybe look is looking
at some review sites or trying to
pull, or maybe it was for the tool.
Yeah, yeah.
For the best SEO tools.
Yeah, the best SEO tools.
It was reviews.
So then some of those review
sites have influence there.
Yeah, totally.
Exactly.
Totally.
But it's so interesting
because like everyone's mind.
I feel like in the last few months
has gone to these like hacks,
like you need schema on your
site, you know all this stuff.
And you're like, yeah, schema.
It never made any txc.
It never made any sense because you're
like, if the LL m's already crawling
the web and organizing information
without schema, why does it need schema?
Or why does it need any of this stuff?
Yeah, I, whatever.
You know, people are stupid and it's a
new territory and there's a lot of fud.
So, um, back to that
gray and yellow graph.
Yeah, this is really interesting.
E, even the gray it, it means that
it's still, like when a new user asks,
it's still gonna run those queries.
They're just the same queries it was
running in the previous seven days.
Is that right?
So, yeah.
Yeah.
The gray means that they, they
will still run these queries.
They were just already captured.
So if the SERP changes for those queries,
it can affect what is influencing it.
Correct.
So there's only two.
There's only two reasons why you would go,
go outside of the band, the initial band.
We can, like the initial band
of different queries space.
The control set.
Yeah, the initial control.
There's three core reasons
why this would occur.
Number one is the training data
or the model changes, right.
You know, basically Google trains
a new data set and I think the, the
recent data set is still like, you
know, till January of last year.
January of 2025, right?
So they're already like a year
and like a few months behind.
So if that changes, then yes, you're
gonna see a large fluctuation in the
same way that a core update would
have a large fluctuation in, in SEO.
Um, so training data changes
or model changes, right?
Gemini, it goes from 3.1 pro to 3.2
pro or flash or whatever, and you're
gonna see some fluctuation there.
So that's like bucket number one.
Bucket number two is that because we know,
right, that all, all these like models are
searching, it's the search changes, right?
So if a core update happens or if
rankings fluctuate for whatever reason,
then yes, you're gonna start to see some
deviation in terms of, you know, what
gets recommended and so on and so forth.
And then the third reason is because the.
It's the LLM itself, right?
Like this, the fresh, the LLM itself
decides that it needs more searching for
this particular thing, which is why you
start to see different shapes, right?
Like this is a commercial prompt.
What are the best SEO tools, right?
And commercial doesn't actually a search
that much and b, deviate that much.
And so you kind of see this more
like, you know, head term to like,
you know, whatever like, um, curve.
Whereas, you know, when, when you
look at, you know, something like,
I'm trying to see if I have like a
what is, yeah, what is a credit SEOre?
It looks like this, right?
It's a lot flatter.
Right, and so the LLM will reason
about what it needs to search for.
And if you just happen to be in a
category where it's like, oh yeah,
I need to do a lot more searches,
like fresh in research, then it will,
you know, like evolve over time.
Those blue histogram graphs, are
you counting the different queries
that it searches across a bunch of
different times where you're a hundred
times that you're asking it, right?
That's right.
Yeah.
Yeah.
And we did that across, you know, many,
like many different problems across
in a single, in a single user's query.
How many times does Gemini seem to
search in a single user's query?
On average is 5.7.
Five point it, it searches
5.7 different queries.
Yes, that's correct.
Whereas chat GBT is like two 0.7.
We have it here.
So chat, GBT on average.
Yeah.
So now this is the more robust dataset.
It searched 77.3% of the time.
When it searched, it only searched
0.77, whereas the average amount
of web searches for Gemini is 5.7.
And that's per response across.
And this is, this is averaged
across a bunch of different like.
Types que like 20,000.
Yeah.
Across informational, fresh, whatever.
Yeah.
Commercial, et cetera.
I, I, I wonder if you did that same.
Study per category?
Like type, type of, right.
Like so you mean software versus
news or politics or something?
For sure.
Or just like, yeah, yeah, yeah, for sure.
Even commercial research
fresh and how that changes.
Yeah, there's like many more.
So yeah, like this 20 to 29%.
It doesn't search the web.
It searches differently.
These emerge, but chat GPT uses the
same two to four zero evolution.
Obviously there's no evolution for
Gemini, like informational, uh, commercial
intent is actually really stable.
That's what we found.
Fresh intent is the most volatile.
That makes a lot of sense.
Um.
This is probably the most interesting
facet that I think just from a pure
like philosophical standpoint, is that
the control run covers 72 to 81% of
like the citations that are likely to
be used, and the control run covers 86
to a hundred percent of the daily brand
mentions that are likely to like occur.
You can see here define,
define control, run.
Again, the control run is like the
initial 100 response sample, and
then we sample it a hundred times.
You know, daily throughout
the next like two weeks.
And then it's like, okay, the control
run basically covered, like the
control runs across both basically
cover the majority of what the model
is likely to recommend and site.
So that kind of goes back to then Rand's
study about like the six to eight brands
will be mentioned the most because
they're likely part of that control run.
Because they're part of the training data.
Yeah, yeah, yeah, exactly.
It's basically the training data,
like guides all, and then the search,
like this is basically saying that
the search component only influences
the, the model, like about 20 to 30%.
That's like what we're seeing.
Right.
Or else it's all just like training data.
Like if you asked it without
any search, any whatever, it's
just gonna always respond with,
within a band of like responses.
That's, and, and the search affects it
by like 20 to 30% is for, for Gemini
and Google products, or ChatGPT or both.
This is because we've, we were, we
like basically stopped doing like chat
GPT because we're just like, well,
chat, GPT doesn't really search much.
And so the conclusion
with chat GPT is twofold.
It's either really easy to
influence it because it, when it
searches, it only searches like.
One to two times, like
as you can see here.
And so then you just need to win the
head ranking and then you influence it.
Or you can make the counter argument
to say, okay, because it only searches
once, there's actually way less surface
area for you to affect what it's
going to like, you know, respond with,
I was asking in a different angle.
I think our, our work with
clients and tracking sort of
them across dozens of clients.
In our pieces through Tracker
suggest to me that chat PT is using
the results of those searches.
It's waiting at less versus its
training data than Google products.
I think that that makes sense.
Google ai, when when we rank clients
with our list posts that everyone
says is dying, but seems to work
extremely well for us and our clients.
When we rank them, we get coverage and
visibility in a IO and perplexity way
faster than chat PT, which suggests to
us that chat PT is just using search
less compared to its training data.
It'll search and in, in your study
is suggesting it, it does a smaller
number of different searches, but
then how much is it weighting that?
And when it's mixing with its
training data, our, our data seems
to suggest it's weight it way less.
Yeah, I would say if I were chat
GPT and I'm only doing one search,
of course I'm gonna wait that way
less than if I'm Gemini and I'm
doing like six to 10 searches.
Yeah.
And like, this is, this is a bit of a
philosophical statement, but it also
makes sense to me with the DNA and
history of those companies, Google
fundamentally was a search company, so of
course they're gonna rely on that more.
Whereas Chachi, bt, I could see why
the DNA of the company would be like.
We will search, but we
don't really need it.
What we built is this really
smart model, and so we're gonna
rely on our training data more.
Right?
Yeah.
Like for this one, it didn't search.
Right?
That's why there's no web
search frequency over here.
Um, which means, but like, yeah.
Yeah.
Go ahead.
I was gonna say, if it doesn't
search, you can't influence it.
Like unless you're
influencing the trading data.
Right.
Yeah.
So anyways, that's, that's kind of what
I wanted to go to is then like, the big
question for, for any business is then
how do you influence the training data?
And you're saying the training
data updates with every model.
The training data does not
update with every model.
It updates independently of models.
And it's extraordinarily expensive.
That's why you see Gemini
three on January of 2025.
Gemini 3.1.
Also January of 2025.
You see all of them are January of 2025.
And so the model, let
me see, January of 2025.
Yeah.
So yeah.
And this one's 2.5 flashlight previews.
Yeah.
So they update differently.
There's a, there's an algorithm
and there's a training
data, and they're different.
Yeah.
My understanding from the OpenAI
perspective is that it's, it's, it, it's
a pretty, it's like black box seat that
OpenAI doesn't reveal what goes in the
training data, how often they update it.
Is that true or do you have any insight?
I, I actually honestly stopped looking so
much at GPT because my, like, my analysis
here and everything points to the fact
that ge, I think Gemini is just gonna win.
Like, I think GPT is
going to like go to like.
Obscure relevance when, when they
move to AI mode is what you're saying?
Like it'll just win or what?
Yeah.
Why?
Why?
It's because they have, they, they
get search, like when, when you search
as a model, your answers are like
anywhere between 10 to like 200% better.
They're just better.
Yeah.
And they get free access to search
and all the Google, like, you
know, everything that Google has.
Yeah.
I mean this is a little bit philosophical
and outside of like work for marketers,
but that's true for all of this stuff.
Which is what?
Is what?
What about this best, this
best for search type queries?
Yes.
For HGBT people use for stuff
completely different from that.
Right, right.
Gimme advice.
But that's, that's like a different,
yeah, that's a different thing.
But I wanna know though though,
like I also just think like
usage of chat GPT is going down.
Yeah, it's going to Claude for workflows
and then Gemini for information retrieval.
I canceled my subscription like last week.
'cause we just started
using Claude for everything.
I'm like, I just don't see a
purpose to chat GPT anymore.
And I think that's, I mean, that's
the case with all these models.
Like, like what Bernard's saying,
I think certain models will
be used for certain use cases.
Like Gemini is clearly the like
productivity app or like the work winner.
And even the lead that we got yesterday,
she came from Gemini and so she's in
Gemini for some purpose for work as well
because it sinks to your Google Drive
and it syncs to all your other products.
Yeah.
It, it, it, it's interesting that
you do think that that's gonna win.
Oh, it's gonna for sure win.
It's gonna for sure win in my opinion.
Like, and so that's why we're
not even really like trying to
build for chat GPT like anymore.
Um.
Anyways, that's just our
own opinions and philosophy.
Okay.
This is like, in my opinion, the
most fascinating component is that
Gemini sources source URLs are
essentially random after day one.
So within sampling, what we find
is that Gemini is actually just
randomly pulling sources from
like a huge bank of possibility.
And what I feel like this.
This means to me as like somebody from
search is that this is basically Google.
Doing user engagement testing, like Google
is basically rebuilding and re rebuilding
responses on the fly using different
sources to then understand whether
or not those responses are, you know,
positive sentiment, good engagement, and
whatever the new rules of LLM engagement.
That's fascinating.
Like, looks like this kind of correlates,
I don't know if it means the same thing
to you, but in the last two months
we've seen so much volatility in.
Ranking positions and just like,
yeah, once we publish something,
it shows up on the first page and
moves down like almost every day.
It's been different.
Yeah.
Is that kind of what you're saying?
They're like testing, like let's say
any, any new piece of content that's
published around the topic, they're now
cycling through a bunch of different
pieces to figure out which one's best.
Yes.
Not which one's best, but which
one's best to be used to construct
or influence the response that
it was gonna give back anyways.
So, you know, like it might
look at your, your source and it
might, we might be talking about.
Whatever top SEO trends and, you know,
maybe the grow and convert talks about,
you know, bottom of funnel and use
case pages and the long, long tail.
And so in that one particular response, it
will look at you and it'll say, long, long
tail pain, point SEO, bottom of funnel.
And then if that user like, oh, tell
me more about this long tail thing.
Tell me more about this.
Oh, I see what you're think.
Then it's like, okay, that
one, it pays attention to that.
Exactly.
That's why this struck a
nerve with the the person.
Yeah.
I mean, this is so interesting
because then it just makes the case
for what I was saying at the very
beginning, where the future has to
be very specific, good information.
Of course like that, that's tailored
to the user and I think like this
whole wave of like use AI to produce
content faster, but it's still just
the same shit that we were producing
before doesn't even really make sense.
You can produce content faster, but
like if we're looking at this, and I
think this is kind of like we're in the
process of writing a post right now just
on like our content strategy for the
future of like AI search and it ties
exactly with what we're seeing here with.
It so it's, it's validating for what
we were thinking, but essentially you
need to write very specific content
around all those various topic areas.
Because I can imagine too, as time goes
on, those waitings might change as the
LLMs figure out when someone searches
this, what do they actually mean by this?
Exactly.
Like, yeah, yeah.
Or, or, or like.
What is the correct strategy or what
do people seem to latch on, latch
onto, like your user engagement signals
that you were just talking about.
You like this video.
Don't forget to subscribe.
You can also get the audio
only versions of these shows
wherever you get your podcasts.
And you can follow us at Grow
and convert.com/newsletter for
any articles and updates for
when these videos come out.