How to do an AI search audit that finds something, and the steps worth skipping

Key takeaways

• Google documents one eligibility rule: a page must be indexed and eligible to appear in Search with a snippet. That is what an audit should check first.
• Ahrefs studied 137,210 domains in June 2026 and found 97% of published llms.txt files were never fetched at all. Auditing for one measures nothing.
• The most useful thing an audit can measure is the gap between being cited and being named, because 61.7% of citations never name the brand.
• Visibility is topic-level, so a site-wide audit score tells you very little about whether you own any particular subject.
• An audit that ends in a vendor setup step is a sales process. Keep the diagnosis separate from the remedy.

A note on where this comes from. We run organic programmes for 250+ enterprises and track more than 10 million prompts across AI engines, and we inherit other people’s audits constantly. The judgements here come from that work. The evidence comes from Google’s documentation and from named, dated 2026 research, and we have kept the two apart.

The short answer

Most AI search audits are long because length looks like rigour. The useful part is short.

Google publishes one eligibility rule: “To be eligible to be shown as a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to be shown in Google Search with a snippet.” It also states that “There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary.”

So a defensible audit has two halves. The first checks whether you clear that documented gate. The second measures what engines currently say about you, which no checklist can tell you and only running prompts can.

Everything else is optional, and some of it is worse than optional.

If you would rather have someone run this with you than work through it alone, you can book a growth audit and we will do the prompt work alongside your team.

What is an AI search audit, and what should it actually check?

An AI search audit is a one-time diagnostic: can engines read you, and what do they currently say about you? It is not the same as ongoing measurement, which is a different job with different instruments. We have written that side up separately in the GEO measurement stack.

A good audit answers four questions in order.

Can an engine retrieve you at all? Indexed, snippet-eligible, text present in the rendered document.

Can it tell who you are? Consistent naming, so references resolve to one organisation.

Does it use you? Citations, which most tools can show you.

Does it name you? A different question, and the one most audits never ask.

The step most audits get wrong

Auditing for an llms.txt file has become standard. It should not have.

In June 2026, Ahrefs examined all 137,210 domains in its Web Analytics product that received traffic in May 2026, checked each root for an llms.txt returning HTTP 200, then analysed every request to those paths. 28% of those domains publish an llms.txt file. Of those files, 97% received zero traffic in May 2026. In the study’s words: “Nothing fetched them at all.”

The breakdown of the few fetches that did happen makes it worse. 96% came from bots, and 77% of those bots were not AI tools at all. AI retrieval bots, the ones that would actually matter, accounted for 1.1% of AI bot requests. Another 12% of fetches came from GEO and AEO tools and researchers, which is the industry studying itself.

One finding ends the argument. “Zero requests came from AI bots for llms.txt files that don’t exist. They never go looking.”

So a missing llms.txt cannot be a finding, because nothing checks for it. Ahrefs’ own conclusion is blunt: “The cons outweigh the pros right now.”

Google is consistent with this. Its documentation states that “You don’t need to create new machine readable files, AI text files, or markup to appear in these features.” The previous version of this page scored 69% of companies as failing on a file that Google says is unnecessary and that almost nothing requests.

How to do an AI search audit: the five checks that survive scrutiny

Ordered by what the documentation supports, then by what the evidence supports, then by judgement. We have marked which is which.

1. Indexation and snippet eligibility. (Documented.) Check robots.txt, noindex, nosnippet, data-nosnippet and max-snippet on your priority templates. A max-snippet value set years ago for an unrelated reason still applies, and it quietly removes you from every AI surface at once. This is the only gate Google states.

2. Text presence in the rendered document. (Documented.) Google’s recommended practices include ensuring “important content is available in textual form”. Open the rendered HTML, not the CMS preview. Gated and PDF-locked material usually fails here, and teams are routinely surprised by how much of their best work is invisible.

3. Naming consistency across owned properties. (Judgement, supported by evidence.) One legal name, one trading name, applied the same way everywhere a machine can read it. If your own site, help centre and careers pages disagree, every downstream signal inherits the confusion. The full version of this is in our entity optimization article.

4. The cited-versus-named gap. (Evidence.) Run your prompt set and record two columns, not one: whether your page was used as a source, and whether your brand name appears in the answer text. Research published in June 2026 by Semrush with Kevin Indig’s Growth Memo, covering 3,981 domain appearances across 115 prompts, 14 countries and four engines, found that 61.7% of citations never named the brand. Only 38.3% of brand appearances included a mention.

5. Topic-level coverage, not a site-wide score. (Evidence.) A July 2026 study of 1,094 US categories and more than 50,000 brands found that only 15.2% of categories had a clear owner, and that domain-level metrics such as authority score predicted topic ownership roughly half the time. A single audit score for your whole site hides the only thing worth knowing: which subjects you are actually recognised for.

The prompt set, which is the audit

Steps one to three take a morning. Step four and five are the audit, and they are the part most checklists replace with a score.

Write 25 real commercial questions your buyers would type. Full sentences, not keywords, because that is how people use these surfaces. Cover definitions, comparisons, alternatives, use cases and buying questions rather than your brand name, since a prompt containing your brand tells you almost nothing.

Run each of those 25 prompts three times, on every engine you care about. Engines are probabilistic. One run is an anecdote, and three runs tell you whether a result is stable.

Record four things per run. Whether your page was cited. Whether your brand was named. Which section of your page was used. Which competitors appeared.

Then wait 90 days before judging any fix. Four worked examples for a mid-market B2B software company:

• “what is data observability and which tools do it well”
• “compare data observability platforms for a mid-size engineering team”
• “what should I look for when evaluating a data observability vendor”
• “how do teams usually roll out data observability across multiple squads”

The competitor column is the one people skip and then wish they had. If three rivals appear in twenty of twenty-five answers and you appear in two, you have a coverage problem, not a technical one. No crawl audit would have told you that.

What the previous version checked, and what it could show

The old page carried 21 statistics. Here is where they came from.

Eleven named no source at all, including the most quotable ones: that 44.2% of citations come from the first 30% of a page, that review platforms carry up to 70% citation weight, and a set of “pages cited” counts for named platforms that we have also removed from another article this fortnight.

Two came from our own benchmark of 110 companies, which we describe publicly and have never published. Our craft rules say a number drawn from unpublished data is not publishable, and this is the fifth time in nine articles we have had to apply that rule.

Two more were attributed to a research source we could not locate. A named attribution that does not resolve to anything is not a source, and it is arguably worse than no attribution, because it looks checked.

One of the eleven is a misreport rather than an invention. The page cited “3,981 measured appearances”, which correctly identifies the Semrush and Growth Memo study, then reported its result as “13.2 percent”. The actual figure is 38.3%. That same wrong number has now appeared on four of our pages.

Our methodology: how we weighted the checks

We scored each candidate audit step on three inputs. The weights decide the order above and the scorecard below.

Documentation carries the most weight because it is the only part of this that is not contested. An llms.txt check scores zero on the first input and zero on the second, which is why it is not in the list.

Audit approaches at a glance

**Where it falls short as a comparison:** the cost row is a shape rather than a quote, because none of this work is standardised. The crawl and schema sweep is genuinely useful for conventional search, and we are rating it only on what it tells you about AI visibility.

What this costs

The documented-gate check costs a morning of someone technical and should be repeated quarterly.

The prompt set is the real expense, and it is mostly time. Twenty-five prompts run three times across four engines is 300 observations, and reading them properly takes longer than generating them. Tooling reduces the labour and does not remove the reading.

Fixes cost whatever the fix costs. Naming consistency is editorial work. Coverage gaps are content commissioning. Neither is an audit expense.

Where Pepper fits

We are not a neutral recommender in this section, and you should read it knowing that.

Pepper works as an organic growth partner rather than a tool vendor. Pepper’s GEO platform runs prompt sets at scale and records citations and mentions as separate columns, which is the step most teams cannot do by hand at volume. Agent Atlas handles the production side once the audit has found the coverage gaps. Customers log in and run it themselves, and a growth team sits alongside for the judgement work.

The framework we run this on is visibility, citability and retrievability. An audit is really a test of all three in sequence.

For a worked example in B2B SaaS, our RVshare case study shows coverage gaps closed after diagnosis rather than before it.

Where it falls short. We cannot give you a single audit score, and we would not trust one. Visibility is topic-level, so one number for a whole domain averages away the finding. We also cannot audit what we cannot read, which means gated content stays a blind spot for us exactly as it does for the engines.

How to choose what to fix first

I will write this in the first person, because an ordering nobody will defend is not worth printing.

Start with the gate, not the content. I have been handed a 40-page audit that scored a client on schema coverage, llms.txt and freshness cadence, and never once checked that the site’s priority templates were snippet-eligible. Two of them were not. The audit was thorough and it missed the only thing that was actually broken.

So my order is: clear the documented gate, confirm the text exists as text, then spend the real effort on the prompt set. Here is the scorecard I would apply to any proposed audit, on a 100 point scale.

Five questions I would ask before paying for one, in this order.

1. “Which of these steps is backed by Google’s documentation?” A stronger answer names the specific guidance. A weaker one says “best practice”, which usually means somebody else’s checklist.
2. “Can I read the actual transcripts?” If the deliverable is a score with no underlying answers, you are buying a number rather than a diagnosis.
3. “How many times was each prompt run?” Once is an anecdote. If the answer is once, the variance is doing the talking.
4. “Does the audit tell me which topics I own?” A site-wide score cannot, because ownership is measured per subject.
5. “What happens at the end of the audit?” If the final step is configuring a product, the diagnosis was never separable from the sale.

Then run the live test yourself before you buy one. Write 25 real commercial questions your buyers would type, in full sentences rather than keywords. Run each of those 25 prompts three times across every engine you care about, because engines are probabilistic and one run is an anecdote. Record four columns: cited, named, which section was used, and which competitors appeared. Give any fix 90 days before judging it. If a supplier cannot describe a process at least this specific, they are selling you a report rather than a diagnosis.

The competitor column is the one teams skip and then wish they had. A stronger audit tells you that three rivals appear in twenty of twenty-five answers while you appear in two. A weaker one gives you a score out of 100 and no idea who is beating you. The first is actionable this quarter. The second is a slide.

Red flags, each one something a supplier actually says. An llms.txt score, when 97% of those files are never fetched. A composite AI visibility score presented as an industry standard. A citation multiplier attached to a format, offered without a study. A benchmark drawn from a dataset you are not allowed to inspect. And our own previous version of this page, which did all four.

And the answer that loses Pepper the sale: if you have already run a 25-prompt set three times across your engines and read the transcripts, you have done the audit, and you do not need us to run one. Spend the budget on the coverage gaps it found.

It comes down to one principle: check what Google documents, read what engines actually say, record cited and named separately, and refuse every score nobody will show their working for, ours included. We published 21 numbers on this page with almost nothing behind them, which is the strongest argument we can make for the principle.

What nobody should promise you

Be sceptical of any of these.

• An llms.txt score. Ahrefs found 97% of those files are never fetched, and AI bots never request one that is missing.
• A single composite AI visibility score. Visibility is topic-level, so one number averages away the finding.
• A universal benchmark. Citation behaviour varies enormously by vertical and query type.
• An audit that ends in a product setup step. That is a sales process with a diagnosis attached.
• A benchmark from a dataset you cannot inspect. Including ours, which is why four numbers came off this page.

Frequently asked questions

What is an AI search audit?
It is a one-time diagnostic that answers two questions: whether AI engines can retrieve and read your pages at all, and what those engines currently say about your brand when real buyers ask real questions.

How long should an AI search audit take?
The technical checks take a morning. The prompt set, which is the part that actually finds things, takes one to two days including reading the transcripts properly rather than skimming a summary score.

Do I need an llms.txt file for AI search?
No. Ahrefs studied 137,210 domains and found 97% of published llms.txt files received zero traffic in May 2026, and Google states you do not need to create AI text files to appear in its AI features.

What should an AI search audit actually measure?
Four things in order: whether you are indexed and snippet-eligible, whether your important content exists as text, whether engines cite you, and whether they name you. The last two are different questions.

How many prompts should I run in an audit?
We use 25 real commercial questions, each run three times on every engine that matters to you. Three runs matter because engines are probabilistic, and a single run cannot tell you whether a result is stable.

Why record citations and mentions separately?
Because they diverge sharply. Research published in June 2026 across 3,981 domain appearances found that 61.7% of citations never named the brand, so counting citations alone overstates how visible you really are.

Can one audit score tell me how I am doing overall?
Not usefully. A July 2026 study of 1,094 categories found only 15.2% had a clear owner, and conventional domain metrics predicted ownership about half the time, so a site-wide number hides the topic-level picture.

How often should I repeat an AI search audit?
Repeat the documented-gate checks quarterly, since templates and directives change. Run the prompt set monthly if you are actively working on coverage, and give any individual fix 90 days before judging it.

Where to go next

• The GEO measurement stack, for ongoing measurement rather than a one-time diagnostic
• Entity optimization and LLM brand recognition, for the naming work in step three
• How to structure content for AI citation, for fixing what the audit finds
• Hidden content and its impact on SEO, for the text-presence question in full
• How to get cited in LLMs
• How we rank and score GEO providers

Sources and further reading

• Ahrefs, “We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read”, Louise Linehan, 15 June 2026. All 137,210 domains in Ahrefs Web Analytics with traffic in May 2026, checked for an llms.txt returning HTTP 200, with every request to those paths classified by response and user agent. Source of the adoption, zero-traffic and bot-composition figures.
• Google Search Central, “AI features and your website”, last updated 10 December 2025. Source of the eligibility rule, the no-additional-requirements statement, the machine-readable-files statement and the textual-form recommendation.
• Semrush with Kevin Indig, Growth Memo, “The ghost citations study”, 9 June 2026. 3,981 domain appearances across 115 prompts, 14 countries and four AI search engines. Source of the 61.7% and 38.3% figures.
• Semrush with Kevin Indig, Growth Memo, “AI visibility is a topic-level game”, 20 July 2026. 1,094 US categories, more than 50,000 brands and over 600,000 citations, tracked monthly from January to June 2026. Source of the 15.2% ownership figure.
• Google Search Central, “Robots meta tag, data-nosnippet, and X-Robots-Tag specifications”, read at source on 5 October 2026. Reference for the noindex, nosnippet, data-nosnippet and max-snippet directives checked in step one.
• Not used, and worth saying why: two figures on the previous version of this page were attributed to a research source we could not locate in any public record. We have treated them as unsourced rather than reproducing them.