Technical

anthropic.com — deep dive · can machines crawl, render & index you
whole-site crawl · live data · 19 Jun 2026
Five tools — the full technical-readability suite
01

The verdict

70TECH / 100

AI can read you — but the house has slow rooms and bare walls.

Server-rendering is flawless (100% SSR — GPTBot and ClaudeBot get real HTML), and the security baseline is solid. But the homepage is a 739KB shell with no schema and no canonical, 18 of 31 pages answer slower than 2s (peaking at 6.3s), only 29% carry structured data, and your safety & research content is under-linked and uncrawlable-fast.

Rendering 100Crawlability 90Mobile 100Security 74Indexability 61Web Vitals 58Architecture 50Structured Data 34
324
checks ·
31 pages

How thorough this audit is

vs SEOptimer's single-page, ~30-check report
Pages crawled
31
SEOptimer: 1
Technical checks run
324
9 per page + 36 site-wide
Oracle technical tools
14
crawler · render · health
URLs in sitemap
475
full index size
Score transparency
8
sub-scores, all shown
Fixes as code
7
drop-in, not "how to"

SEOptimer grades one URL with black-box letter grades (it stamped anthropic.com's careers page a B, GEO an F — without explaining either). We crawl the whole site, show the math behind every sub-score, scan each page for AI-readability and injection payloads, and hand back the repair code. Same instrument, an order of magnitude deeper — and honest about what only your server logs can answer.

02

The math — every sub-score, shown

Technical Health 70 = weighted sum of 8 measured sub-scores

no black box
Rendering / AI-readabilityweight 16 — SSR 100%, AI bots get real HTML
100
Crawlability & discoveryweight 14 — robots ✓, sitemap 475 URLs ✓, llms.txt 404
90
Mobile & usabilityweight 8 — viewport 100%, favicon ✓, responsive
100
Security & technologyweight 10 — HTTPS+Brotli+CSP+DMARC ✓; HSTS 1h, headers missing
74
Indexabilityweight 12 — canonical 61%, 8 heading-skip pages, dup meta
61
Core Web Vitalsweight 15 — home LCP 4.9s poor, CLS ~0, secondaries ~2s
58
Architecture & internal linksweight 12 — 0 orphans, but 6 deep pages ≤3 inbound
50
Structured dataweight 13 — schema on 29% of pages, FAQPage-only
34

Weights sum to 100; each sub-score is computed from real crawl + header/DNS + Lighthouse lab data — every input is in the per-page table and the sections below. Owner-only signals (server-log crawl frequency, true field-CrUX per page, full index coverage) are excluded from the score, not faked.

03

Crawlability & discovery — can the bots find everything

AI & search crawler access

1 bot blocked

Your robots.txt allows everything and points to one sitemap — so on paper every crawler is welcome. But a live per-bot probe (9 bots × 3 URLs) caught what robots-parsing misses: Bingbot is blocked at the edge — 403 on every request from the Cloudflare WAF, not robots. That silently cuts Bing's index, which grounds Bing Copilot. The other gap: zero crawler-specific rules — you neither manage crawl budget nor explicitly welcome the AI bots that now mediate discovery.

User-Agent: *
Allow: /

Sitemap: https://www.anthropic.com/sitemap.xml
GPTBot · OAI-SearchBot
allowed (default)
ClaudeBot · Claude-Web
allowed (default)
PerplexityBot · Google-Extended
allowed (200)
Googlebot
allowed (200)
Bingbot
blocked (403 — WAF)

Discovery files

XML sitemap475 URLs · fresh lastmod
Sitemap referenced in robots1 sitemap
llms.txt404 — absent
AI-crawler directivesnone (all default)
Sitemap is flat (not index)475 entries direct

The sitemap lists 475 URLs — your real index footprint — yet only a handful carry schema or canonical (below). And there's no llms.txt: the file Claude's own ecosystem reads, missing from Anthropic's own domain.

04

Indexability — clean signals to the index

Canonical

61%

of pages declare a canonical. The other 39% — including the homepage — leave canonicalization to Google's guess.

Heading structure

8

pages skip a heading level (e.g. H1→H3). 2 pages (/engineering, /claude-corps) have no H1 at all; H1 present on 94%.

Duplicate signals

/responsible-scaling-policy was crawled twice (two URL forms), and 8 pages share boilerplate meta — duplicate signals that dilute the index.

05

AI rendering — what the bots actually see

Server-side rendering — your strongest plane

SSR 100%

All 31 crawled pages serve real HTML before JavaScript runs. This matters more than anything else here: GPTBot, ClaudeBot and PerplexityBot do not execute JavaScript — a client-rendered page returns an empty <div id="root"> and is invisible to them. Anthropic passes this gate cleanly, which is why it gets cited at all.

Server-rendered
31/31
real HTML to bots
Avg words in HTML
3,084
extractable text
NOARCHIVE / NOSNIPPET
none
no silent citation kill
Oracle render signals
4
SSR · clean-tree · noarchive

Oracle signals scored here: sig-ssr-mandate · sig-noarchive-kill · sig-clean-accessibility-tree · sig-copilot-prometheus — all PASS. This is the one area where Anthropic is built right; every other section is upside.

06

Architecture — where the link equity flows

Internal-link graph

31-page sample
Orphan pages
0
all reachable
Avg inbound links
24.4
per page
Avg outbound
48
nav-heavy

No orphans is good — but the average hides the problem: the nav links the same ~24 pages everywhere, while your deepest content gets almost nothing.

The under-linked: your defining content is starved

equity gap
PageInbound linksStatus
/news/the-anthropic-institute1starved
/responsible-scaling-policy2starved
/news/core-views-on-ai-safety2starved
/features/81k-interviews2starved
/news/claude-opus-4-83
/research, /company, /careers (nav)30hub

Your Responsible Scaling Policy, Core Views on AI Safety and the Anthropic Institute — the pages that define your authority on safety — get 1–3 internal links each, vs 30 for routine nav pages. Google reads that as "not important." Add contextual links from the hubs.

07

Core Web Vitals & speed

Core Web Vitals — homepage

LCP poor
LCP
4.9s
poor (>4s)
CLS
0.00
good
TBT (INP proxy)
0ms
good

Lighthouse lab pass (desktop). The homepage's 4.9s LCP is the drag — a 739KB page with a slow largest-element render. Secondary pages (/product/*) land near 2.0s. CLS and interactivity are clean.

Server response time (TTFB)

18 slow pages
Pages > 2s TTFB
18 / 31
58% of the site
Slowest
6.3s
/engineering
Avg TTFB
2.3s
across 31 pages

Despite Cloudflare in front, most content pages answer slowly from the Webflow origin: /engineering 6.3s, /product/enterprise 5.6s, /claude/fable 5.4s, /careers 5.0s. The fast pages (home 129ms, /events 87ms) prove the CDN works — the origin/cache config is the issue.

08

Structured data — the machine-readable layer

Schema coverage 29% — and it's the wrong schema

34 / 100

Only 9 of 31 pages carry any structured data, and the only types present are FAQPage / Question / Answer. There is no Organization schema (the entity anchor every AI engine reads), no Article on news/research, no Person for authors, and no BreadcrumbList. The product pages have schema; the homepage, research, news and policy pages have none.

Pages with schema
9 / 31
29%
Schema types found
3
FAQPage only
Organization schema
0
the entity anchor
Article / Person
0
on 6,000-word essays

Oracle signals here: sig-schema-three-plus · sig-chunk-schema-combo · sig-faqpage-schema — measured lift from full structured data is +30–40% (Stackmatix, Grade-B). The 30,524-word Constitution and the 9,875-word 81K-interviews study run with zero Article markup — invisible structure to every engine.

09

Security & technology stack

Transport & headers

74 / 100
HTTPS + valid certCloudflare
Brotli compressioncontent-encoding: br
HTTP/3 supportedalt-svc: h3
Content-Security-Policypresent
HSTSmax-age 1h (want 1yr)
X-Frame-Optionsmissing
X-Content-Type-Optionsmissing
Referrer-Policymissing

DNS & email authentication

strong
Hosting / CDN
Cloudflare · 160.79.104.10
Nameservers
randy/isla.ns.cloudflare.com
Mail (MX)
Google Workspace
SPF
v=spf1 …google.com ~all
DMARC
p=reject (strict)

Email auth is strong — DMARC p=reject with reporting. One tweak: SPF ends in ~all (softfail); move to -all (hardfail) once you've confirmed all senders. The three missing response headers are one Cloudflare Transform Rule away.

10

Defensive scan — AI prompt-injection (IDPI)

Indirect Prompt-Injection / AI-poisoning scan

31 pages clean

Every crawled page is scanned for content engineered to hijack an AI that reads it — the December 2025 Unit 42 attack class. anthropic.com is clean: no hidden-text or injection payloads found across 31 pages. No competing AEO/SEO tool runs this scan — it's a defensive layer unique to this audit, and it matters most for the company whose models are the target.

Zero-sized text
font-size:0 · line-height:0
clean
Off-screen text
left:-9999px
clean
Transparency tricks
opacity:0 · color-match
clean
HashJack fragments
URL #payloads
clean
11

Per-page forensic — top issues first

Every page, ranked by issue count

31 pages · click to expand all
PageIssuesTTFBKBSSRSchemaCanon.H1SkipsFlags
/policy-on-the-ai-exponential44.1s138011ttfb
/transparency42.1s612010
/ (homepage)4129ms739011
/engineering36.3s140000no H1 6.3s
/learn34.1s105010ttfb
/news/expanding-project-glasswing33.6s125010
/news/core-views-on-ai-safety32.8s226010
/claude-corps3469ms198000no H1
/constitution3376ms685010
/news/claude-opus-4-83226ms202011

The split is stark: the two clean pages are products (/product/claude-code, /product/claude-cowork) — schema, canonical, sub-300ms. Everything that tells Anthropic's story (homepage, research, news, safety, the Constitution) runs bare and slow. Same template, opposite discipline.

12

The fixes — drop-in code, impact-ordered

CRITICAL

Add Organization schema site-wide (the entity anchor)

+30–40% AI-citation

Zero Organization schema today. This single block, in <head> on every page, is what AI engines read to resolve "Anthropic" as an entity.

<script type="application/ld+json">
{
  "@context":"https://schema.org","@type":"Organization",
  "@id":"https://www.anthropic.com/#org",
  "name":"Anthropic","url":"https://www.anthropic.com/",
  "logo":"https://www.anthropic.com/images/logo.png",
  "foundingDate":"2021",
  "sameAs":[
    "https://en.wikipedia.org/wiki/Anthropic",
    "https://www.wikidata.org/wiki/Q116758847",
    "https://x.com/AnthropicAI",
    "https://www.linkedin.com/company/anthropicresearch",
    "https://www.youtube.com/@anthropic-ai"
  ]
}
</script>
CRITICAL

Fix the two pages with no H1, then the heading skips

indexability

/engineering and /claude-corps render with zero H1; 8 pages skip a heading level. Give every page exactly one H1 and no level jumps (H1→H2→H3).

HIGH

Publish llms.txt + add a self-referencing canonical

discovery

llms.txt 404s and 39% of pages (incl. the homepage) have no canonical. Ship both:

# /llms.txt
# Anthropic — AI safety and research
> Claude, Constitutional AI, MCP, the Responsible Scaling Policy.
- [Research](https://www.anthropic.com/research)
- [Claude](https://www.anthropic.com/claude)
- [Responsible Scaling Policy](https://www.anthropic.com/responsible-scaling-policy)

<!-- + on every page -->
<link rel="canonical" href="https://www.anthropic.com/{path}">
HIGH

Cache the origin — kill the 2–6s TTFB on content pages

18 pages

18 pages answer >2s (peaking 6.3s) from the Webflow origin while the CDN sits idle. Add a Cloudflare Cache Rule for HTML on these paths (the homepage already proves 129ms is possible).

MEDIUM

Add the three missing security headers + extend HSTS

security 74→90

One Cloudflare Transform Rule closes clickjacking + MIME-sniff + referrer leakage and makes HSTS preload-eligible.

Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Referrer-Policy: strict-origin-when-cross-origin
MEDIUM

Internal-link your safety & research content from the hubs

equity

Add contextual links to /responsible-scaling-policy, /news/core-views-on-ai-safety and /news/the-anthropic-institute (1–3 inbound today) from /research, /company and /policy (30 inbound). Resolve the duplicate /responsible-scaling-policy URL to one canonical form.

13

Methodology & honesty

Crawl
31 pages · native fetch + render probe
Core Web Vitals
Lighthouse lab pass
Headers / compression
Node https (no auto-decompress)
DNS / email auth
Node resolver (SPF/DMARC live)
robots / sitemap / llms
public fetch + parse
Internal-link graph
re-fetched 31 URLs → inbound

100% real, zero defaults. Every value above is measured from anthropic.com on 19 Jun 2026. Honest gaps: orphan detection is within the 31-page sample (a full crawl needs the 475-URL sitemap); per-page CWV is lab not field-CrUX; true crawl frequency, render-budget and full index coverage need your server logs + Search Console — the one place SEOptimer and Semrush also go blind, and the upgrade path. SPF/DMARC, headers, robots, sitemap and the link graph are all live, not estimated.

14

Go deeper — the technical-readability suite

Each tool is a full screen on real anthropic.com data, with transparent sub-scores and deployable fixes.

BLISS·OPTIMIZER — Technical · anthropic.com · 19 Jun 2026 Health 70 MODERATE · SSR 100% · schema 29% · 324 checks × 31 pages · LCP 4.9s