AI crawlers / ledger

The AI crawler ledger

Sites on AIPayGen publish AI Terms: what AI may do with their pages and the price, stated before any payment. AI companies read most sites for free. Here are the counts: AI crawlers shown our own site's terms and price (live), visits to the other sites running our snippet or gateway (whole days, through yesterday), and how many paid.

On aipaygen.com

aipaygen.com, liveLast 7 daysLast 30 days
AI crawler visitsThe same count as our live log46,70446,704
Asked to pay (HTTP 402)Shown the terms and a price; only marketplace listings are priced, our own pages stay free to read6,2166,216
Used a crawler's name from the wrong addressBy name below109109

On other sites running our snippet or gateway

Sites whose owner proved the site to us, combined, counted in whole days (UTC) through 2026-11-21. Shown once 5 or more sites report.

All other sites, combinedLast 7 days, through 2026-11-21Last 30 days, through 2026-11-21
AI crawler visits countedShown once 5 or more sites report (n < 5 now)HiddenHidden
Used a crawler's name from the wrong addressShown once 5 or more sites report (n < 5 now)HiddenHidden

Updated 2026-11-22 13:19 UTC, refreshed at most every 5 minutes. Totals and crawler names only: no site, page, wallet or day is shown. No page was paid for in the last 30 days on aipaygen.com, or through our gateway in the 30 days through 2026-11-21.

Fake crawlers by name

A request that names one of these 13 crawlers from an address its company doesn't publish is counted as fake. This under-counts on purpose: only crawlers whose company publishes its addresses can be checked, so a fake Bytespider or meta-externalagent is never counted.

On aipaygen.com, liveLast 7 daysLast 30 days
Claude-UserNot from Anthropic's published addresses3535
GPTBotNot from OpenAI's published addresses2020
ChatGPT-UserNot from OpenAI's published addresses1010
OAI-SearchBotNot from OpenAI's published addresses99
Perplexity-UserNot from Perplexity's published addresses77
DuckAssistBotNot from DuckDuckGo's published addresses77
CCBotNot from Common Crawl's published addresses77
PerplexityBotNot from Perplexity's published addresses66
ClaudeBotNot from Anthropic's published addresses44
Claude-SearchBotNot from Anthropic's published addresses44
On other sites, combinedLast 7 days, through 2026-11-21Last 30 days, through 2026-11-21
Shown once 5 or more sites report (n < 5 now).

How these are counted

Is a crawler in your own logs real? Check one address or a whole log, in your browser. To count them on your site and set your terms, start the free report, no wallet needed.