AI crawlers / ledger
The AI crawler ledger
Sites on AIPayGen publish AI Terms: what AI may do with their pages and the price, stated before any payment. AI companies read most sites for free. Here are the counts: AI crawlers shown our own site's terms and price (live), visits to the other sites running our snippet or gateway (whole days, through yesterday), and how many paid.
On aipaygen.com
| aipaygen.com, live | Last 7 days | Last 30 days |
|---|---|---|
| AI crawler visitsThe same count as our live log | 46,704 | 46,704 |
| Asked to pay (HTTP 402)Shown the terms and a price; only marketplace listings are priced, our own pages stay free to read | 6,216 | 6,216 |
| Payments for pages, by AI crawlers or agents | 0 | 0 |
| Used a crawler's name from the wrong addressBy name below | 109 | 109 |
On other sites running our snippet or gateway
Sites whose owner proved the site to us, combined, counted in whole days (UTC) through 2026-11-21. Shown once 5 or more sites report.
| All other sites, combined | Last 7 days, through 2026-11-21 | Last 30 days, through 2026-11-21 |
|---|---|---|
| AI crawler visits countedShown once 5 or more sites report (n < 5 now) | Hidden | Hidden |
| Payments for pages, by AI crawlers or agents, through our gatewayA total of 0 always shows. Shown once 5 or more sites report (n < 5 now) | 0 | 0 |
| Used a crawler's name from the wrong addressShown once 5 or more sites report (n < 5 now) | Hidden | Hidden |
Updated 2026-11-22 13:19 UTC, refreshed at most every 5 minutes. Totals and crawler names only: no site, page, wallet or day is shown. No page was paid for in the last 30 days on aipaygen.com, or through our gateway in the 30 days through 2026-11-21.
Fake crawlers by name
A request that names one of these 13 crawlers from an address its company doesn't publish is counted as fake. This under-counts on purpose: only crawlers whose company publishes its addresses can be checked, so a fake Bytespider or meta-externalagent is never counted.
| On aipaygen.com, live | Last 7 days | Last 30 days |
|---|---|---|
| Claude-UserNot from Anthropic's published addresses | 35 | 35 |
| GPTBotNot from OpenAI's published addresses | 20 | 20 |
| ChatGPT-UserNot from OpenAI's published addresses | 10 | 10 |
| OAI-SearchBotNot from OpenAI's published addresses | 9 | 9 |
| Perplexity-UserNot from Perplexity's published addresses | 7 | 7 |
| DuckAssistBotNot from DuckDuckGo's published addresses | 7 | 7 |
| CCBotNot from Common Crawl's published addresses | 7 | 7 |
| PerplexityBotNot from Perplexity's published addresses | 6 | 6 |
| ClaudeBotNot from Anthropic's published addresses | 4 | 4 |
| Claude-SearchBotNot from Anthropic's published addresses | 4 | 4 |
| On other sites, combined | Last 7 days, through 2026-11-21 | Last 30 days, through 2026-11-21 |
|---|---|---|
| Shown once 5 or more sites report (n < 5 now). | ||
How these are counted
- Visits are counted by each site's own snippet, plugin or our gateway, which check the crawler's name and, where its company publishes them, its addresses. A site can let some crawlers in free (search crawlers, a free allowance, or count-only while it decides), so not every visit is priced.
- Other sites count only once their owner has proved the site to us, and each site counts once toward the 5, however many setups it has.
- Asked to pay is recorded for aipaygen.com only, for now.
- Payments count settled payments for pages. They can't tell an AI crawler from an AI agent paying for a page, so they count both. Our own wallets and our weekly test purchases are left out.
- Visit counts are kept for 90 days. Every number here is for the last 7 or the last 30 days, as its column says. Also as JSON.
Is a crawler in your own logs real? Check one address or a whole log, in your browser. To count them on your site and set your terms, start the free report, no wallet needed.