The enterprise SEO audit framework: 47 checks for sites at scale

The short answer

An enterprise SEO audit is a structured review of a large site that turns findings into an engineering roadmap. At scale the useful question is not which category a finding belongs to. It is what you fix to resolve it: a system, a template, or a single page. These 47 checks are grouped that way, because a template fix lands on fifty thousand URLs and a page fix lands on one.

Key takeaways

• Group findings by unit of fix, not by category. Systems, templates, then the handful of pages that carry the programme. This is the difference between a roadmap and a spreadsheet.
• The threshold everyone quotes is wrong. Enterprise is not “100,000 URLs”. Google’s own crawl budget guidance names large sites as 1 million or more pages changing weekly, or 10,000 or more changing daily. Size times change rate is the test, not size.
• Start with logs, not a crawler. A crawl tells you what a bot could reach. Logs tell you what Googlebot and the AI crawlers actually fetched. The gap between those two is where enterprise problems live.
• Four items on most enterprise checklists are things Google says are unnecessary, including llms.txt and AI-specific rewrites. Cutting them frees engineering time for the checks that matter.
• Pepper is an agentic organic growth engine and an organic growth partner. Agent Atlas runs the recurring checks in this framework, Pepper’s GEO platform reports Brand Visibility and Domain Prompt Presence against them, and a growth team works the roadmap alongside your engineers.

A note on where this comes from. My lane is what breaks inside organisations rather than what looks tidy in a deck. Enterprise audits break in a specific way: the findings are correct and nobody ships them. Pepper runs organic for more than 250 enterprises over eight years and tracks more than 10 million prompts across every major engine. That shapes this framework, along with the audits we inherit from other providers and the engineering standups where a 200-row spreadsheet quietly dies.

What is an enterprise SEO audit, and how is it different at scale?

An enterprise SEO audit is a structured diagnostic of a large, complex site covering crawl access, indexation, architecture, template implementation, content systems and authority signals, which then converts every finding into a prioritised roadmap with owners.

The definition is ordinary. What changes at scale is the economics of a fix.

On a 200-page site a finding is a task. On a 500,000-page site it is either a template change worth making fifty thousand times, or a page change worth making once. Those two belong in different queues, sized differently and owned by different people. Grouping them by category, which is what almost every checklist does, is why enterprise audits produce documents rather than deployments.

So this framework asks one question of every finding: what is the unit of fix?

If the surrounding vocabulary is new, our glossary of core AEO terms covers it, and the wider argument for running organic as a function sits in the enterprise playbook.

What actually counts as enterprise, according to Google

Every checklist on this topic opens with a URL count, usually 100,000, and none of them cites anything. Google publishes real thresholds, and they are different.

Google’s crawl budget guidance says the advice applies to “Large sites (1 million+ unique pages) with content that changes moderately often (once a week)”, to “Medium or larger sites (10,000+ unique pages) with very rapidly changing content (daily)”, and to “Sites with a large portion of their total URLs classified by Search Console as Discovered – currently not indexed”.

Read that carefully, because it changes who should be reading this.

• Size alone is not the test. A million static pages can need less crawl management than ten thousand that change daily.
• The third condition is the useful one. If a large share of your URLs sit in “Discovered, currently not indexed”, you have a crawl budget problem regardless of your page count.
• So the real trigger is size multiplied by change rate, plus what Search Console already tells you.

Where this falls short: these thresholds are Google’s, for Googlebot. No other engine publishes an equivalent, so treat them as the best available proxy rather than a universal rule.

If you want your own crawl reality read against this before you commit engineering time, book a growth audit and we will run it with your logs.

How we weighted the 47 checks

Every check carries a severity weight, fixed before any audit is scored, so two auditors reach the same roadmap order. Weights sum to 100.

Blocking beats everything else for a reason rather than a preference. A Stanford-led evaluation published in May 2026 found that retrieval failures drive over 70% of errors when engines answer questions, not reasoning failures. If the machine cannot reach the page, nothing else you fix on it counts.

The 47 checks at a glance

## The 47 checks

Numbered continuously so you can cite “check 31” in a ticket. Every group closes with where it falls short.

Systems: crawl and log evidence, checks 1 to 8

Start here, always. A crawl shows what a bot could do. Logs show what it did.

1. Pull server logs for at least 30 days, segmented by user agent, and confirm the sample covers your full URL space rather than one subdirectory.
2. Compare crawled URLs against your sitemap set. Anything crawled and absent from the sitemap is a discovery path you did not intend.
3. Identify the top 100 most-crawled URLs. If parameters, filters or search results dominate, you have found your crawl waste before opening a single report.
4. Measure crawl frequency against change frequency per template. Pages that change daily and get crawled monthly are a business problem, not a technical one.
5. Check the ratio of 200s to 3xx, 4xx and 5xx in bot traffic, by template.
6. Confirm Googlebot fetches render-critical resources. Blocked CSS or JavaScript in logs means what you see is not what it sees.
7. Separate real bots from spoofed ones by reverse DNS before you draw any conclusion about volume.
8. Record server response time for bot requests specifically. Google states that if it can load pages faster, it “might be able to read more content from your site”.

Where it falls short: log access is the single most common blocker on an enterprise audit, and it is political rather than technical. If you cannot get logs in week one, escalate it in week one.

Systems: indexation and crawl budget, checks 9 to 16

9. Read the Page Indexing report and quantify the share of URLs in “Discovered, currently not indexed”. Google names a large share of these as its own trigger for crawl budget work.
10. Eliminate soft 404s. Google is explicit that these “will continue to be crawled, and waste your budget”.
11. Consolidate duplicate content so crawling concentrates on unique content rather than unique URLs, which is Google’s own framing.
12. Audit faceted navigation for combinatorial URL explosion, and decide per facet whether it should be crawlable, linkable, both or neither.
13. Find and kill session identifiers and tracking parameters in internal links.
14. Test for infinite spaces: calendars, filters, pagination without bounds, and internal search results exposed to crawlers.
15. Verify canonical logic per template rather than per page, and confirm canonicals are self-referential where they should be.
16. Check that your sitemaps are segmented, current and under the size limits, with lastmod values that are actually true.

Where it falls short: these are the checks most likely to produce a finding that is correct and unshippable, because faceted navigation usually belongs to a commerce or product team with its own roadmap. Bring them in before you write the recommendation, not after.

Systems: architecture and internal linking, checks 17 to 23

17. Measure click depth from the homepage to your revenue pages, by template. Anything past four hops at scale is effectively hidden.
18. Find orphan pages that exist in the sitemap but receive no internal links.
19. Audit the internal link modules themselves, because at scale links are generated by components rather than written by hand.
20. Check anchor text generated by templates describes the destination rather than repeating a category name thousands of times.
21. Verify pagination and infinite scroll expose a crawlable path to everything behind them.
22. Confirm hreflang is consistent and reciprocal if you run multiple locales, and that it is not fighting your canonicals.
23. Map the site’s entity structure: consistent naming of products and categories, so an engine can tell that ten thousand pages belong to one brand.

Where it falls short: architecture findings are the slowest to ship and the easiest to argue about, because they touch navigation that other teams own commercially. Expect a quarter, not a sprint.

Systems: AI crawler access, checks 24 to 29

The 2026 additions, and the ones most enterprise checklists still omit. High leverage, because they are configuration rather than construction.

24. Read robots.txt for AI crawler rules explicitly. A blanket block added years ago for scrapers now removes you from AI answers. Our guide to robots.txt for AI crawlers covers the specifics.
25. Distinguish index crawlers from on-demand fetchers. Blocking a scheduled crawler removes you from an index. Blocking a live fetcher breaks citation of a page you are otherwise happy to rank. PerplexityBot is the clearest worked example.
26. Confirm AI crawlers can render your answers, not just fetch your HTML shell, which is the same server-side rendering question as check 6 with a different set of bots. See crawlability for AI.
27. Check CDN and WAF rules for bot mitigation that silently blocks AI user agents your robots.txt allows.
28. Quantify AI crawler share of total bot traffic from the same logs, so you know whether this is a real channel for you yet. Our guide to crawl budget for AI bots sets out how.
29. Set up the free first-party citation report in Bing Webmaster Tools before you buy any monitoring, so you have a baseline that costs nothing.

Where it falls short: only Google and Microsoft publish anything resembling documentation here, so several of these checks rely on your own log evidence rather than a vendor’s stated behaviour.

Templates: on-page at scale, checks 30 to 37

At scale you do not audit titles. You audit the function that generates titles.

30. Review the title generation logic per template, and check the output at the edges: missing fields, very long product names, empty categories.
31. Do the same for meta descriptions, and confirm no template produces duplicates across thousands of URLs.
32. Check every template has exactly one H1 and that it is populated from a field that is never empty.
33. Verify heading hierarchy is generated in order, with no skipped levels introduced by a component.
34. Confirm each template front-loads its answer rather than opening with navigation or boilerplate.
35. Test question-shaped headings where the template supports them, because a heading a model can match to a query is what gets lifted. The method sits in how to structure content for AI citation.
36. Measure the boilerplate-to-unique-content ratio per template. At scale this is the difference between thin and useful.
37. Check image alt text generation produces something descriptive rather than a filename or an empty string.

Where it falls short: template fixes are cheap to specify and slow to release, because they sit in a product backlog behind revenue features. The multiplier is what wins that argument: one change, fifty thousand URLs.

Templates: structured data, checks 38 to 42

38. Validate structured data per template rather than per page, and check the failure cases as well as the happy path.
39. Confirm Organization markup is present and consistent sitewide. Organization schema is the entity anchor.
40. Check Article or Product markup matches the visible content on the page, because mismatches are a spam signal rather than a bonus.
41. Verify FAQPage markup only appears where a real FAQ exists, and never on generic questions added to hit a template.
42. Confirm no template over-optimises markup in ways Google’s own guidance calls unnecessary. Our view on what still earns its place is in schema markup.

Where it falls short: structured data is necessary for rich results and entity clarity, and Google says plainly it is not required for generative AI search. Do not let anyone sell it to you as a citation lever.

Pages: the few that carry the programme, checks 43 to 47

Five checks, applied to the pages that actually earn, not to the long tail.

43. Identify the pages producing revenue or pipeline, and audit only those by hand. On most enterprise sites this is a few dozen URLs.
44. Check each one says something a model could not reconstruct from three competitors’ pages.
45. Confirm each carries original material: data, a method, a number nobody else has.
46. Audit the earned authority pointing at them, because third-party sources carry most of the citation opportunity. Independent May 2026 analysis of more than 25 million cited links put earned media at 84% of AI citations.
47. Verify each page is measured individually, with cited URLs and prompt-level visibility rather than a site-wide average. That is what citation rate is for.

Where it falls short: this group is where the value is and where audits spend the least time, because it is qualitative and cannot be crawled. Budget real editorial hours for it.

Four checks to delete from your enterprise checklist

Now the part that gives engineering time back, and it comes from Google rather than from us.

Google’s guidance states, verbatim:

• On machine-readable files, including llms.txt: “You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search.”
• On chunking: “There’s no requirement to break your content into tiny pieces for AI to better understand it.”
• On writing for machines: “You don’t need to write in a specific way just for generative AI search.”
• On markup: “Structured data isn’t required for generative AI search, and there’s no special schema.org markup you need to add.”

At enterprise scale none of those is a small waste. Chunking a 500,000-page site is a platform project. Deleting it from the audit is worth more than most findings you would add.

Then hold the other half, because a literal reading overshoots. Google describes Google. Structured data still earns its place for rich results and entity clarity, other engines publish no equivalent guidance, and we keep schema on client sites. We stopped selling it as an AI citation lever.

How to run the audit, in order

Sequence matters more than coverage, because the groups depend on each other. Audit templates before you know what bots fetch and you are auditing pages nobody reaches.

The ordering rule is simple. Evidence first, then the fixes that multiply, then the fixes that do not.

What an enterprise SEO audit costs

Three costs, and the first is the one people quote.

• The audit itself. Published SEO and GEO pricing we verified in this category runs from $3,000 a month for an ongoing programme, with consulting rates published at $150 to $300 an hour. Most providers quote after a call, so a standalone audit is usually scoped rather than listed.
• The engineering to ship it. This is the real cost, and it almost never appears in a proposal. A template change is hours to specify and weeks to release. Architecture work is quarters.
• The measurement to prove it. Free at the baseline, since Bing Webmaster Tools reports your Copilot citations at no cost. Paid platforms add engines and prompt volume on top.

Pepper does not publish pricing, so budget discovery with us is a conversation rather than a page. What we will say plainly is that the audit is the cheap part. If nobody has engineering capacity to ship the roadmap, buy the capacity before you buy the audit.

How Pepper runs this framework

Pepper is an agentic organic growth engine and an organic growth partner. For a framework this long, the relevant question is who keeps it running after week six, and that is the half most audits leave open.

• Agent Atlas runs the recurring checks. Agents are workflows rather than a chat box. The log comparisons, crawl deltas, indexation shifts, template validation and schema checks in this framework are exactly the work that should run on a schedule rather than in a quarterly document. System agents are fixed, user agents are editable and versioned, and customers log in and build and run their own agents in Atlas.
• Pepper’s GEO platform reports the outcome. Brand Visibility for how often engines mention you, Domain Prompt Presence for how often they cite a page from your domain, and Share of Voice for your slice of the category. The gap between the first two tells you whether your remaining constraint is authority rather than architecture. See the platform.
• A growth team works the roadmap with your engineers. Agents for scale, experts for judgement, one team on the hook for the number. That is the difference between a finding and a deployment.
• Proof rather than adjectives. Acceldata went from 85 to more than 300 top-three keywords with 6X organic traffic growth, and one hero guide carried over 260,000 impressions. Work of exactly this shape produced it.

Where it falls short: we do not publish pricing, so budget discovery is a conversation rather than a page. We are built for teams running organic as a long-term function with an internal team to work alongside, so a one-off audit document with nobody to ship it is not what we are for.

How to choose what to fix first

I would not start at check 1 and work down. I would sort every finding by the severity weighting above, then by whether anybody can ship it this quarter. In the audits we inherit, the top of the roadmap is usually correct and undeliverable.

The framing judgement first, anchored outside my own opinion. Google’s guidance says optimising for generative AI search “is optimizing for the search experience, and thus still SEO”. Its crawl budget documentation also gives real thresholds rather than a round number. Read that as permission to stop treating AI search as a separate audit. Then hold the other half: Google describes Google, and no other engine publishes equivalent documentation, so your log evidence is the only cross-engine truth you have.

Here is the 100-point scorecard I would run over any audit before committing engineering time, including one of ours.

Then run the live test, on us as readily as on anyone else. Give the auditor 25 questions drawn from your own site. Ask for each answer with evidence attached, not a methodology. For example: “which templates waste the most crawl budget”, “what share of our URLs are discovered and not indexed”, “which AI crawlers can render our product pages”, “which of our pages earn citations today”.

Ask them to come back with five things. Which findings block retrieval entirely. Which multiply across templates. Who owns each fix. What breaks first if nothing changes. What exactly they would ship in the next 90 days. An auditor answering with log evidence and a sequenced roadmap beats one answering with a crawl export. And one who cannot name the owner of a faceted navigation fix has not audited an enterprise site.

The weaker way to run this, and what most enterprise audits do: run a crawler across the whole domain, export everything, group by category, colour-code by severity, present 200 rows. Every step is real work. None of it survives an engineering backlog, because nothing in it says which single change would fix ten thousand URLs.

The stronger sequence: get logs, quantify crawl waste, then fix what blocks retrieval. Ship template changes in multiplier order, hand-audit the few pages that earn, and instrument all of it so the next audit is a diff rather than a rebuild. It matters because at scale the winning move is never finding more issues. It is finding the one template change worth making fifty thousand times.

Red flags, each one something I have seen in a real enterprise audit:

• A 200-row spreadsheet with no unit of fix against any row.
• Findings from a crawler with no log evidence, so nobody knows whether bots ever reached the pages.
• A severity column with no stated weighting, which makes the order unfalsifiable.
• llms.txt or content chunking as a recommendation, both of which Google says are unnecessary.
• No owner named for anything outside the marketing team.
• A recommendation to fix the long tail by hand at a scale where that is arithmetically impossible.
• A guarantee of rankings, which Google itself advises against, because no third party has access to the ranking systems.

Five questions I would ask, and what a good answer sounds like.

1. “Which of these findings blocks retrieval entirely?” A good auditor names two or three immediately and puts everything else behind them.
2. “Show me the log evidence for this finding.” If the answer is a crawler export, the finding is a hypothesis.
3. “What is the unit of fix, and who owns it?” Every row should answer this without a follow-up meeting.
4. “What would you not fix?” An auditor who recommends everything has not prioritised anything.
5. “What does the next audit look like?” The right answer is a diff against this one, run continuously, not another full rebuild in six months.

If I reduce this to one principle: audit the systems and templates that repeat, and hand-audit only the pages that earn. Everything else at enterprise scale is a spreadsheet.

The honest closing note, and it costs us something. Very few providers are equally strong across log analysis, template engineering, editorial judgement and measurement, ourselves included, and the good ones will tell you which is weakest. Ask, and treat a straight answer as a positive signal.

What nobody should promise you

Nobody should promise a ranking or an AI citation from an audit. Google advises against providers who guarantee rankings, because no third party has access to the ranking systems.

Nobody should promise that a document changes anything. An audit with no engineering capacity behind it is a description of a problem, and we would rather tell you that before you commission one.

Nobody should promise a single composite score for site health, or a universal benchmark for crawl efficiency. Your own trend against your own baseline is the only comparison that holds, which is the argument in what actually matters in AI search measurement.

Where this stops working, including for us

If your site is under a few thousand URLs and changes rarely, this framework is heavier than you need. Run the crawl and indexation groups, skip the template economics, then spend the time on content.

If nobody can ship engineering work this quarter, you do not need an enterprise audit yet and you do not need us. Fix the access problems in checks 24 to 29, which are configuration rather than construction, and revisit when you have release capacity.

Where Pepper falls short: covered in our own section above rather than buried here. We do not publish pricing, and we are built for a long-term organic function with an internal team alongside, so a one-off audit document is not what we sell.

Where to go next

Get 30 days of logs this week. Everything in this framework that matters depends on them, and the request usually takes longer than the analysis.

Then run the four AI crawler access checks, because they are hours of configuration with the highest leverage in the list. If you want the shorter version to start with, our 7-point AI search audit is the one-page entry point to this framework, and the AI search operating system covers how to run it as a function once the audit is done. To see where you stand across engines first, see where you show up.

Frequently asked questions

What is an enterprise SEO audit?
A structured diagnostic of a large, complex site covering crawl access, indexation, architecture, template implementation, content systems and authority, converted into a prioritised engineering roadmap with named owners rather than a list of findings.

How many URLs make a site enterprise?
Google’s crawl budget guidance names large sites as 1 million or more pages changing weekly, or 10,000 or more changing daily, plus any site with a large share of URLs in “Discovered, currently not indexed”. Size times change rate is the test.

How often should an enterprise SEO audit run?
Treat the full framework as annual and the systems groups as continuous. Log comparisons, indexation shifts and crawler access checks should run on a schedule, so the next audit is a diff against the last rather than a rebuild.

What should an enterprise SEO audit include?
Log evidence before crawler output, indexation and crawl budget analysis, architecture and internal linking, AI crawler access, template-level on-page and structured data, and hand audits of only the pages that earn revenue.

Why do enterprise SEO audits fail to get implemented?
Because findings are grouped by category rather than by what you fix. Engineering cannot size a category. It can size a template change, which is why every finding needs a unit of fix and an owner attached.

Should an enterprise audit include AI crawler checks?
Yes, and most checklists still omit them entirely. Checks 24 to 29 cover robots rules, index crawlers against on-demand fetchers, rendering and CDN blocking. They are configuration rather than construction, so they are the cheapest and highest-leverage items in the framework.

Do I need llms.txt for an enterprise site?
Google states you do not need machine readable files, AI text files, markup or Markdown to appear in Google Search. No engine documents a benefit, so it is not worth platform time at enterprise scale.

What is the difference between a crawl and a log analysis?
A crawl shows what a bot could reach if it tried. Logs show what bots actually fetched, how often, and what they got back. The gap between the two is where most enterprise problems are hiding.

Sources and further reading

• Google Search Central, large site owner’s guide to managing crawl budget. Source of the 1 million pages weekly and 10,000 pages daily thresholds, the “Discovered – currently not indexed” trigger, the soft 404 and duplicate consolidation guidance, and the server response quotation. Product documentation, current until Google changes it.
• Google Search Central, guide to optimizing for generative AI features, and the announcement post, 15 May 2026. Source of the four items to delete and the “still SEO” framing. Applies to Google Search only.
• Suzgun, Shen, Bianchi, Spangher, Icard, Ho, Jurafsky and Zou, “Evaluating Commercial AI Chatbots as News Intermediaries”, arXiv:2605.22785, submitted 21 May 2026. Source of the finding that retrieval failures drive over 70% of errors, which is why blocking carries the heaviest severity weight. Limitation: news questions on commercial chatbots, not enterprise sites.
• Muck Rack, “Earned media still drives 84% of AI citations”, What is AI reading? May 2026 edition, published 7 May 2026. Sample: more than 25 million cited links across ChatGPT, Claude and Gemini in 17 industries. Cited for check 46.
• Microsoft, AI Performance in Bing Webmaster Tools, public preview announced 10 February 2026. The free citation baseline named in check 29.
• Pepper, Acceldata case study. Source of the 85 to more than 300 top-three keywords and 6X organic growth figures, verified on the page. One account, not a benchmark.

What is not here. No industry benchmark for how many findings a typical enterprise audit produces, or what share get shipped, because we found no primary research measuring either and will not invent one. The 47 checks and their grouping are our framework, and the figures say so rather than implying they were measured. Vendor pricing referenced in the cost section was read on 8 September 2026 and moves often.