The verdict — can the engines crawl, render & trust this site
A whole-site technical audit across 31 crawled pages, scored on 8 measured sub-scores that sum, with their weights, to a single number — no black box. The headline: Site Health 70 / 100, MODERATE. The foundations an AI crawler needs are excellent — 100% server-rendered, every bot allowed — but the page is bleeding on speed, structured data and security headers. Every issue below names the exact pages and ships the fix.
A health score is only honest if you can see the math. Every sub-score below shows its weight, its value, and the one-line reason — and every issue points at the pages it lives on.
The math — Site Health 70 = the weighted sum of 8 measured sub-scores
no black box · tap any sub-score—
Issues by severity — what to fix, and where it lives
— critical and — warnings across the crawl, grouped by type and sorted by severity. Each card names the affected pages — this isn't "improve your site," it's "fix these exact URLs." — pages are already perfectly clean.
Crawlability & discovery — can every bot find everything
The front door. Before anything can rank or be cited, the crawlers — search and AI — have to be let in and pointed at the content.
AI & search crawler access
all allowedrobots.txt is a permissive User-Agent: * / Allow: / — every crawler below is allowed. The nuance: there are no bot-specific rules at all, so AI access rides entirely on the wildcard. That works today; it's worth making the AI allow-list explicit.
Discovery files
AI rendering — what the bots actually see
Server-side rendering — your strongest plane
SSR 100%All 31 crawled pages serve real HTML before JavaScript runs. This matters more than anything else on this page: GPTBot, ClaudeBot and PerplexityBot do not execute JavaScript — a client-rendered page returns an empty <div id="root"> and is invisible to them. Anthropic clears this gate cleanly, which is why it gets cited at all. Protect it: every framework change should be re-checked for SSR.
Indexability — clean signals to the index
Crawlable isn't the same as cleanly indexable. Three signal-dilution problems repeat across the crawl.
Architecture — where the link equity flows
Internal links are how authority moves through a site. The structure is sound at the top — zero orphans, strong hubs — but the defining content is starved.
Top hubs — equity sources
inbound links, 31-page sampleThe under-linked — starved pages
equity gapYour flagship safety & research writing — the Responsible Scaling Policy, Core Views on AI Safety, the Anthropic Institute — sits at 1–3 inbound links. The exact content that should define the entity is the hardest for crawlers to reach.
Speed & Core Web Vitals — site-wide
The most measurable ranking signals Google ships — and there are two truths most tools only half-show: the field data is what Google actually ranks on (real users), the lab data is the diagnostic that tells you where to optimize. We show both, and recommend the fix for each.
Core Web Vitals — field data
real users · 28-day CrUX · the ranking signalSupporting metrics
diagnostic — not directly rankedThe lab diagnostic — where the headroom is
—Page weight by type
——
Optimization opportunities — lab diagnostics
with the recommended fixThe three Core Web Vitals — thresholds, causes & fixes
your value vs the bandsPer-page response & weight — all 31 crawled pages
—Per-page, the crawl gives two continuous speed signals — server response (TTFB) and HTML weight; the full Core Web Vitals are origin-level (the cards above) plus a desktop-lab pass on 3 templates. Thresholds — TTFB ≤800ms good / ≤1.8s / >1.8s poor. Field TTFB (real users) is a healthy 752 ms; these crawl numbers are cold render waits behind the CDN cache. Click any page for its full vitals — field, lab, crawl signals and issues.
Across monitored reference sites in early 2026, pages that pass Core Web Vitals show a 3–4× higher AI-citation rate than failing pages in the same topic. Not a published ranking factor — but the mechanism is plausible: fast, well-structured pages are more reliably crawled, parsed and trusted by AI answer engines. Speed is no longer just a Google-organic play; it's table stakes for being quotable.
Structured data — the machine-readable layer
The lowest sub-score, and the highest-leverage fix on the site — because it's the one change that moves three planes at once (technical, on-page E-E-A-T, and entity/GEO).
Schema coverage 29% — and it's the wrong schema
34 / 10022 of 31 pages carry no structured data at all. The 9 that do carry only FAQPage markup — there is no Organization, Article, Product, Person or Breadcrumb schema anywhere on the domain. That's the entity-proof an answer-engine needs, missing.
sameAs identity graph, authorship, and breadcrumbs — extractable verbatim. Drop-in code in §12.Security & technology stack
Transport and email authentication are strong; the gap is a set of missing HTTP response headers — cheap, edge-level one-liners that also signal engineering rigor.
Transport & response headers
74 / 100DNS & email authentication
strongServed behind Cloudflare with HTTP/3 and Brotli. Email is DMARC p=reject (strongest) — but SPF ends in a soft ~all; tighten to -all.
Defensive scan — AI prompt-injection (IDPI)
Indirect prompt-injection / AI-poisoning scan
31 pages cleanAs AI crawlers read your pages, hidden text can hijack them. We scanned all 31 pages for the December-2025 Unit-42 payload patterns — zero-size text (font-size:0), off-screen positioning (left:-9999px), transparency tricks (opacity:0, color-on-background), and HashJack URL-fragment payloads. All clean. No rival SEO audit even looks for this — it's the defensive half of being AI-readable.
Per-page crawl health — every page, worst first
The whole crawl, ranked by issue count. Click any page for its full technical breakdown and the fixes it needs. 2 pages (/product/claude-code, /product/claude-cowork) are flawless — proof the platform can ship clean pages.
| Page | Issues | Status | TTFB | Weight | Schema | Canon | H1 | Skips |
|---|
Deployable fixes — drop-in code, impact-ordered
The repair, shipped. These are edge-level and template-level changes — most ship in one deploy.
① Security response headers — Cloudflare / edge
# lock down the response — all currently missing or weak
/*
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Referrer-Policy: strict-origin-when-cross-origin
Permissions-Policy: geolocation=(), microphone=(), camera=()
② Self-referential canonical — 12 pages incl. homepage
<!-- emit the page's own absolute URL; also de-dupes --> <!-- /responsible-scaling-policy, currently crawled via two URL forms --> <link rel="canonical" href="https://www.anthropic.com/responsible-scaling-policy">
③ llms.txt — the AI-readable site map (currently 404)
# Anthropic — AI-readable index > Anthropic builds Claude, a family of frontier AI models. ## Core - [Research](https://www.anthropic.com/research): safety & interpretability - [Claude models](https://www.anthropic.com/claude): Opus, Sonnet, Haiku - [Responsible Scaling Policy](https://www.anthropic.com/responsible-scaling-policy)
Fix priority
impact-orderedOrdered by health-score lift × ease. The structured-data block (§08) is listed first because it's the only fix that also moves the On-Page and GEO planes — one deploy, three scores.