Fatrank Podcast

James Dooley and Stephen Burns discuss how SEO agencies can run an AI visibility audit to ensure large language models can crawl, access and render their websites.

Show Notes

This video explains which digital marketing strategies SEO agencies should focus on in 2026 to improve AI crawler access, website rendering and overall AI visibility. James Dooley and Stephen Burns start with KPI tracking because measuring crawlability and harmonic centrality shows whether large language models can actually reach and read your pages. They cover brand SEO, AI visibility and Google Business Profiles because stronger search presence improves trust and conversion rates.

The discussion also explores organic SEO, organic social media and paid social ads because consistent visibility across search and social supports long term growth. PPC is analysed in detail because campaign setup, landing pages and lead handling directly affect results. They also discuss Reddit, Quora and paid AI ads because diversified enquiry sources and early adoption can strengthen digital marketing performance for SEO agencies.

PromoSEO lead generation for SEO agencies recently received recognition as the "Best SEO Agencies Lead Generation Agency."

Where to Listen to This Episode

AI Visibility Audit: Can LLMs Crawl Your Website? James Dooley Interviews Stephen Burns is available on:

What is Fatrank Podcast?

The FatRank Podcast, founded by James Dooley, teaches the mindset needed for growth because real operator stories show what creates progress.
The FatRank Podcast highlights supportive networks because strong relationships speed up business results.
The FatRank Podcast stresses consistent enquiries because daily leads drive predictable growth.
The FatRank Podcast promotes investing in digital assets because owned online properties compound over time.

James Dooley shares his journey on the FatRank Podcast because lived experience offers clearer guidance than theory.
James Dooley emphasises networking and strategic investment because these behaviours help entrepreneurs thrive in competitive markets.

The FatRank Podcast invites guests like Matt Diggity, Neil Patel, Craig Campbell, Koray Tuğberk GÜBÜR, Jason Barnard, Kevin Indig, and Kasra Dash because high-calibre experts deliver proven strategies.
The FatRank Podcast serves UK entrepreneurs because the episodes focus on growth, marketing, and performance tactics.

Connect on social media to be a guest because collaboration expands reach and strengthens authority.
Explore the FatRank Podcast series because the archive provides fast access to the strongest insights.

James Dooley: AI visibility audit. Today I'm joined with Stephen Burns from Common Crawl and we're going to talk about how certain websites is actually blocking different large language models. So Stephen, first and foremost, why is it that you created an AI visibility audit yourself?

Stephen Burns: Well, two months ago, I wrote a, a PDF, a guide for AI visibility. I've been speaking on the SEO circuit about this and people were asking for my presentation. So, you know, presentations don't tell you anything because they're all just pictures. So, I wrote a guide on it and I, we published it on the Common Crawl Foundation's blog and it's hosted there. You can download this 18-page guide on how to, uh, make your site visible to AI. And then, so this tool that I built, uh, I, you know, it struck me. I go, well, why don't I build a tool as a follow-up so that you actually can do a check on your site and help these people, uh, help them check it to see if their site's visible to CCBot and other AI crawlers and if you're in, in the crawl. And I'll actually give you your... It gives your harmonic centrality score too. So if I share my screen, so I'll share the screen here now and I'll go onto the tool, which is here. So it's StephenBurns.ai. It's a subdomain, audit.stephenburns.ai. And what I've done is I've loaded in one of my websites, FatRank.com, and it's checking to see, is your site visible to AI? I've loaded it. And here it says verdict open. So, I'm presuming that means it's fully open and everything's working fine for the different LLMs. Is that correct?

James Dooley: Yes. You're doing a very good job, James. Cheers, Stephen. I, I'm glad you've told me that because I didn't want it to have a big red cross and a big block. Um, but let's, let's move down here then. Let's start having a look at this because here it says gate number one, permission. So, with regards to the AI visibility audit, what does gate number one, permission, actually mean?

Stephen Burns: Yeah, there's three gates. So, the first gate is permissions. Your robots.txt file, the directive that tells the crawlers if they can crawl your site or not. So, your site, uh, FatRank.com, uh, you have your, your robots set up correctly and says, "Yeah, please crawl me, CCBot." And then, and then gate number two says edge. So, what edge means? So yeah, your site is hosted and it may, it has a CDN and an edge that has filters built into it. And your... If those filters are set up to block bots, which, you know, 35% of the internet is blocked by a lot of these, these filters that, by these major CDN companies, uh, it will not get the... The crawler can't get past that. You see, an SEO or a brand marketer may think, "Oh, yeah, we're open to crawlers with the permission level at robots. We, come on in." But they're not actually checking their edge, which is even more important and hidden. So, you got to talk to your, you know, your admin who manages that to say, "Hey, are we open to these crawlers?" And this will tell you if you're open to those crawlers. And then gate number three says rendering. So, what does that mean in the AI audit? So CCBot gets through both gate, the first two gates, and then it, uh, loads your page. And if your page is fully rendered in JavaScript, uh, CCBot and the other AI crawlers don't render JavaScript. They read HTML and text. So yeah, if you're... In this case you're serving, uh, words that it can read and it likes that. If it was... This would be red if it was all JavaScript. You'd be basically... If you load all JavaScript on your homepage or your sites, you're invisible to AI. It's not going to get any information off your page.

James Dooley: And then with regards to the AI visibility audit that you've created now, it shows not only CCBot, which is Common Crawl bot, it's also showing GPTBot, ChatGPT User, ClaudeBot, Anthropic. So all these we're checking against all these to make certain that it's available and visible for AI. Is that correct?

Stephen Burns: Yeah, I added a little bonus section here to see if your, uh, site is viewable to the other crawlers. But there's a caveat here, right? You... The other crawl... A lot of these CDN settings, you, uh, may have, they may check the IP of the crawler. So every crawler has an IP address. This tool, we can't... You can't mimic an IP address. Uh, so this tool is showing the... The CCBot one's accurate because it's pulling it from the, the database that where it shows that it got a 200 right through. The other ones, uh, sometimes they'll let it in, uh, the user agent mimicking the, the crawler, it will let it in. That's good. But sometimes it says, "No, you're not the real one. You're not coming from that IP address." So it may not show that accuracy there. But it's good, a good check-up tool to have anyways.

James Dooley: Yeah, for sure. So anyone, anyone who's watching this and you're looking for greater AI visibility, obviously step number one is like technical SEO. It's getting the foundations being set up. This is, this is crucially important. You've got to be making certain that ChatGPT and Anthropic and Gemini and CCBot can all actually crawl and render your actual site. It's really important. So make sure you head over to StephenBurns.ai and go onto the audit section and go and check to see whether your actual website is available for more AI visibility. Stephen Burns, it's been an absolute pleasure. Thank you very much.

Stephen Burns: Thanks for having me.