AI crawlers / YouBot

YouBot: what it is and how to block or charge it

YouBot is run by You.com. You.com's crawler for its AI search and assistant.

Block YouBot in robots.txt

User-agent: YouBot
Disallow: /

robots.txt is a request, not a lock: well-behaved crawlers follow it. To block by user agent at the server, match YouBot in the User-Agent header.

Or charge YouBot per page

Instead of blocking it, answer YouBot with HTTP 402 Payment Required and a price. Crawlers and agents that support x402 pay a fraction of a cent in USDC straight to your wallet and get the page; people and search engines browse free. Set it up free on pay-per-crawl in about five minutes.

Is YouBot reading your site?

See the full list of AI crawlers. Facts here follow each operator's own documentation; tell us if one changes.