Agentic Web CensusCheck your site
EN

September 2026 · Lighthouse 13.4.0

Is your site ready for AI agents?

One URL, one report: what an AI agent can read on your site, what guides it, what it can act on — scored the same way as the census below, so the two compare.

The Agentic Web Census Score

Every figure over time

Poor21.0%

AWCS

What the AWCS is, and how it is measured

Ours, not Google’s. Lighthouse scores its own category at 58.4% for the same month; we do not publish that as ours, because it weighs a rendering metric at half and its weights are conditional and undocumented.

Measured on 158,165 pages, 95.3% of which carried the agentic category, with a confidence margin of ±0.13.

Pages an agent can parse 53%  +  Sites with a valid llms.txt 27%  +  Sites exposing tools 20%
      ×  Agents that can reach the site
the formulaThree parts add up and the fourth multiplies, because it is a precondition: an agent that cannot fetch the page can do nothing with how readable the page is.

▲ +0.9
AWCS over 5 months

15.2%21.0%

Read

What Read measures

Lighthouse builds the accessibility tree an agent would traverse and scores the page pass or fail. The denominator is only the pages the audit could evaluate: a notApplicable or an error is excluded rather than counted as a failure, and the fraction that could be evaluated is published beside it as the coverage.

n=158,168 · coverage 95.3% · margin ±0.13 pp.

53% of the score
Read on its full scale

Weak32.2%

Read: 5 months, up▲ +1.3 pp
Guide

What Guide measures

The llms-txt audit passing: a file is served AND Lighthouse considers it valid. The denominator is every sampled page, so this is a fraction of the whole web and not of the sites that have a file. This is the figure the index uses, because a broken file does not guide an agent.

n=158,165 · coverage 95.3% · margin ±0.13 pp.

27% of the score
Guide on its full scale

Poor14.8%

Guide: 5 months, up▲ +0.9 pp
Act

What Act measures

A count, not a percentage: how many sampled sites register WebMCP tools that an agent can call. It is the audit that defines the Act dimension — the other two WebMCP audits measure how good those tools are, not whether they exist.

n=158,176 · coverage 95.3% · margin ±0.13 pp.

20% of the score
Act on its full scale

Very poor4.6%

Act: 5 months, up▲ +0.2 pp
Reachable

What Reachable measures

Read from the robots.txt of the sampled sites: of the AI agents we track, how many are NOT disallowed. Gradual and not a yes or no — turning away one agent out of seventeen is not the same as turning away all of them. A site with no robots.txt blocks nobody, so it counts as fully reachable: that is a measurement, not a gap. This is the only figure that MULTIPLIES the index instead of adding to it, because it is a precondition: if an agent cannot fetch the page, nothing else about the page matters. One caveat we would rather state than hide: a blanket Disallow under User-agent: * is not counted, because the corpus-wide measurement cannot see it either, and we would rather be consistent than clever.

n=166,024 · coverage 100.0% · margin ±0.13 pp.

multiplies the rest
Reachable on its full scale

Excellent95.5%

Reachable: 5 months, no significant change= −0.1 pp

Technologies

All 752 technologies

How to read these figures

The colour says what a bar IS, not how it scores: the accent is the one this page is about, grey is everything it is compared with, and the hollow bar is the web overall — the ruler, not a rival. Whether a figure is above or below the rest of the web is in the number and in the arrow beside it.

Every figure here comes from a sample, not from counting every site — so it carries a margin: how much it could move if we had happened to sample different pages. A figure measured on a thousand pages has a wide margin; one measured on a hundred thousand has a narrow one. It matters for one thing: two figures that differ by less than the wider of their margins are not one above the other, they are tied. That is why nothing here has a position number.

Three of the four dimensions add up. The fourth multiplies: how much of the site is reachable by the AI agents we track. It works that way because it is a precondition, not an improvement — if an agent cannot fetch the page, its accessibility tree does not matter — so a site that turns away every agent scores zero however good the rest of it is. Blocking used to EARN points here, under the reasoning that having decided was a sign of maturity. In an index that measures how ready a site is for AI agents, that was backwards. And it is not a value judgement: readiness is a matter of fact, and whether you want to be ready is your call. A low score here means you chose to be closed, not that you did something wrong.

The census also measures analytics tags, payment processors and protocol features, whose figure describes their customers rather than themselves — which is why they are not on this list.

What each category of technology is worth to an agent. Best first.

CategoryAverage indexTechnologies
Static site generator

Static site generator

Technologies
4
Average index
28.0%
Margin
±4.88
The web overall
21.0%

These technologies shape the site, so the figure says something about the technology.

Static site generator averages 28.0%28.0%
4
JavaScript frameworks

JavaScript frameworks

Technologies
33
Average index
22.8%
Margin
±4.23
The web overall
21.0%

These technologies shape the site, so the figure says something about the technology.

JavaScript frameworks averages 22.8%22.8%
33
CDN

CDN

Technologies
24
Average index
21.0%
Margin
±5.27
The web overall
21.0%

These do not shape the site, so the figure describes their clientele: sites that take card payments are better built, which is not the processor’s doing.

CDN averages 21.0%21.0%
24
UI frameworks

UI frameworks

Technologies
18
Average index
20.9%
Margin
±4.63
The web overall
21.0%

These do not shape the site, so the figure describes their clientele: sites that take card payments are better built, which is not the processor’s doing.

UI frameworks averages 20.9%20.9%
18
PaaS

PaaS

Technologies
19
Average index
20.7%
Margin
±4.76
The web overall
21.0%

These technologies shape the site, so the figure says something about the technology.

PaaS averages 20.7%20.7%
19
Ecommerce

Ecommerce

Technologies
18
Average index
20.0%
Margin
±5.01
The web overall
21.0%

These technologies shape the site, so the figure says something about the technology.

Ecommerce averages 20.0%20.0%
18
CMS

CMS

Technologies
34
Average index
19.4%
Margin
±4.86
The web overall
21.0%

These technologies shape the site, so the figure says something about the technology.

CMS averages 19.4%19.4%
34
Page builders

Page builders

Technologies
21
Average index
18.6%
Margin
±5.17
The web overall
21.0%

These technologies shape the site, so the figure says something about the technology.

Page builders averages 18.6%18.6%
21
Web frameworks

Web frameworks

Technologies
16
Average index
18.6%
Margin
±4.45
The web overall
21.0%

These technologies shape the site, so the figure says something about the technology.

Web frameworks averages 18.6%18.6%
16
Web servers

Web servers

Technologies
15
Average index
18.4%
Margin
±4.82
The web overall
21.0%

These technologies shape the site, so the figure says something about the technology.

Web servers averages 18.4%18.4%
15
Hosting

Hosting

Technologies
26
Average index
17.4%
Margin
±4.74
The web overall
21.0%

These technologies shape the site, so the figure says something about the technology.

Hosting averages 17.4%17.4%
26
Programming languages

Programming languages

Technologies
9
Average index
17.2%
Margin
±5.22
The web overall
21.0%

These do not shape the site, so the figure describes their clientele: sites that take card payments are better built, which is not the processor’s doing.

Programming languages averages 17.2%17.2%
9

Markets

All 108 suffixes

What this axis can and cannot tell you

It cannot tell you about governments or universities: the suffixes that reliably identify an institution are small populations, and none of them reaches the publication threshold. And a suffix is not a sector — a .com can be anything, so nothing here should be read as an industry.

The same score, by domain suffix.

SuffixAWCSMonth by month
.com

.com

Sample
n=65,770
Coverage
95.0%
Margin
±0.21
The web overall
21.0%

Two figures closer together than ±0.21 are tied, not ranked.

.com: 23.1%23.1%
.com: 5 months, up▲ +1.1
.co.uk

.co.uk

Sample
n=3,108
Coverage
96.6%
Margin
±0.95
The web overall
21.0%

Two figures closer together than ±0.95 are tied, not ranked.

.co.uk: 22.5%22.5%
.co.uk: 5 months, no significant change= +0.7
.com.br

.com.br

Sample
n=4,147
Coverage
92.8%
Margin
±0.82
The web overall
21.0%

Two figures closer together than ±0.82 are tied, not ranked.

.com.br: 21.4%21.4%
.com.br: 5 months, up▲ +2.8
.org

.org

Sample
n=6,110
Coverage
96.5%
Margin
±0.68
The web overall
21.0%

Two figures closer together than ±0.68 are tied, not ranked.

.org: 20.5%20.5%
.org: 5 months, up▲ +2.0
.de

.de

Sample
n=5,536
Coverage
97.6%
Margin
±0.71
The web overall
21.0%

Two figures closer together than ±0.71 are tied, not ranked.

.de: 20.3%20.3%
.de: 5 months, no significant change= +0.5
.net

.net

Sample
n=3,523
Coverage
95.2%
Margin
±0.89
The web overall
21.0%

Two figures closer together than ±0.89 are tied, not ranked.

.net: 19.7%19.7%
.net: 5 months, no significant change= −0.7
.jp

.jp

Sample
n=3,058
Coverage
97.0%
Margin
±0.96
The web overall
21.0%

Two figures closer together than ±0.96 are tied, not ranked.

.jp: 16.5%16.5%
.jp: 5 months, up▲ +1.3
.ru

.ru

Sample
n=4,050
Coverage
96.6%
Margin
±0.83
The web overall
21.0%

Two figures closer together than ±0.83 are tied, not ranked.

.ru: 10.2%10.2%
.ru: 5 months, no significant change= +0.7

AI agents

All 17 agents

Who is named in a robots.txt, and whether the rule lets them through.

AgentSites naming itMonth by month
GPTBot

gptbot

Sample
n=166,024
Coverage
100.0%
Margin
±0.13 pp
The web overall
21.0%

Two figures closer together than ±0.13 pp are tied, not ranked.

GPTBot: 10.0%10.0%
GPTBot: 5 months, up▲ +0.4 pp
ClaudeBot

claudebot

Sample
n=166,024
Coverage
100.0%
Margin
±0.13 pp
The web overall
21.0%

Two figures closer together than ±0.13 pp are tied, not ranked.

ClaudeBot: 9.2%9.2%
ClaudeBot: 5 months, up▲ +0.6 pp
CCBot

ccbot

Sample
n=166,024
Coverage
100.0%
Margin
±0.13 pp
The web overall
21.0%

Two figures closer together than ±0.13 pp are tied, not ranked.

CCBot: 8.5%8.5%
CCBot: 5 months, no significant change= +0.1 pp
Google-Extended

google-extended

Sample
n=166,024
Coverage
100.0%
Margin
±0.13 pp
The web overall
21.0%

Two figures closer together than ±0.13 pp are tied, not ranked.

Google-Extended: 8.4%8.4%
Google-Extended: 5 months, up▲ +0.3 pp
Bytespider

bytespider

Sample
n=166,024
Coverage
100.0%
Margin
±0.13 pp
The web overall
21.0%

Two figures closer together than ±0.13 pp are tied, not ranked.

Bytespider: 8.0%8.0%
Bytespider: 5 months, no significant change= +0.1 pp
Amazonbot

amazonbot

Sample
n=166,024
Coverage
100.0%
Margin
±0.13 pp
The web overall
21.0%

Two figures closer together than ±0.13 pp are tied, not ranked.

Amazonbot: 7.9%7.9%
Amazonbot: 5 months, no significant change= +0.1 pp
meta-externalagent

meta-externalagent

Sample
n=166,024
Coverage
100.0%
Margin
±0.13 pp
The web overall
21.0%

Two figures closer together than ±0.13 pp are tied, not ranked.

meta-externalagent: 7.6%7.6%
meta-externalagent: 5 months, up▲ +0.3 pp
Applebot-Extended

applebot-extended

Sample
n=166,024
Coverage
100.0%
Margin
±0.13 pp
The web overall
21.0%

Two figures closer together than ±0.13 pp are tied, not ranked.

Applebot-Extended: 7.5%7.5%
Applebot-Extended: 5 months, up▲ +0.1 pp

Reports

All 5 reports

Each month frozen, so a citation stays true.

MonthAWCSMonth by month
September 2026
September 2026: 21.0%21.0%
September 2026: 5 months, up▲ +0.9
August 2026
August 2026: 20.1%20.1%
August 2026: 4 months, up▲ +0.2
July 2026
July 2026: 19.9%19.9%
July 2026: 3 months, up▲ +2.5
June 2026
June 2026: 17.4%17.4%
June 2026: 2 months, up▲ +2.2
May 2026
May 2026: 15.2%15.2%
—