⏺ webmcp-ai.dev Get the audit →

home / blog / analysis

Cloudflare’s Agent Readiness Score, explained

Cloudflare scanned 200,000 sites and found only ~4% pass its agent content check. Here's what the Agent Readiness Score actually measures — discoverability, content accessibility, bot access, capabilities — and how to fix each category.

Updated 2026-08-28 · webmcp-ai.dev research desk

Cloudflare's Agent Readiness Score did for the agentic web what PageSpeed did for performance: it gave the industry a number to argue about. Cloudflare scanned around 200,000 websites and the headline finding was brutal — only about 4% passed the content-accessibility check. If you've run your site through it and stared at a low score, here's what it's actually measuring, and what to do about each part.

The categories, decoded

1. Discoverability

Can agents find and map your site? Checks center on robots.txt and an XML sitemap. This is table stakes — but note the modern twist: robots.txt is no longer just about Googlebot. Agents look at how you treat GPTBot, ClaudeBot, Google-Extended, PerplexityBot and friends. A blanket block, often inherited from a 2023-era "block AI" template, silently removes you from agentic search and recommendations.

2. Content accessibility

The category almost everyone fails. The question: can a machine get your content as clean text without executing a JavaScript bundle? Signals include markdown content negotiation (serving a text version when an agent asks for it), server-rendered HTML, and llms.txt. Client-side-only SPAs fail hardest here — to most agents they are blank pages.

3. Bot access control

Not "block all bots" but "manage them deliberately": signals like Content-Signals policies and Web Bot Auth support, distinguishing agents you welcome from scrapers you don't. Scoring well here means having made explicit decisions, in machine-readable form.

4. Capabilities — where WebMCP lives

The forward-looking category, and the one that separates "readable" sites from usable ones. Cloudflare's scanner renders your page headlessly and checks whether it registers WebMCP tools via document.modelContext, and looks for capability discovery surfaces: an MCP server card at /.well-known/mcp/server-card.json, API catalogs (RFC 9727), and OAuth discovery metadata. Very few sites score anything here — which is exactly why the ones that do get outsized attention from agents (and from posts like this one).

How to raise your score, in order of effort

  1. Hours: fix robots.txt (deliberate AI-crawler rules + sitemap reference), add canonical tags, publish llms.txt.
  2. A day: ensure your core pages render content server-side; add Schema.org JSON-LD for your business entities.
  3. Days: register 3–5 WebMCP tools around your highest-value actions — see our tool patterns cookbook — and join Chrome's origin trial.

Where our scan differs

Our free scanner covers the same territory with two differences worth knowing. First, weighting: we put 30% of the score on the capabilities layer, because that's where the competitive gap is opening. Second, output: Cloudflare tells you what failed; our deep audit is built to tell you what to build — including the specific WebMCP tools that fit your business, as ready-to-implement code. Run both; they agree far more than they differ, and the differences are informative.