AI crawlers / OAI-SearchBot
OAI-SearchBot: what it is and how to block or charge it
OAI-SearchBot is run by OpenAI. OpenAI's crawler for showing and linking websites in ChatGPT search results.
Indexes pages for AI search answers
- Run by
- OpenAI
- Its name in your logs
OAI-SearchBot- Published addresses
- 36 ranges. Check an address
- Same company
- GPTBot, ChatGPT-User
Block OAI-SearchBot in robots.txt
User-agent: OAI-SearchBot
Disallow: /robots.txt is a request, not a lock: well-behaved crawlers follow it. To block by user agent at the server, match OAI-SearchBot in the User-Agent header.
Or charge OAI-SearchBot per page
Instead of blocking it, answer OAI-SearchBot with HTTP 402 Payment Required and a price. Crawlers and agents that support x402 pay a fraction of a cent in USDC straight to your wallet and get the page; people and search engines browse free. Set it up free on pay-per-crawl in about five minutes.
Is OAI-SearchBot reading your site?
See the full list of AI crawlers. Facts here follow each operator's own documentation; tell us if one changes.