AI crawlers / meta-externalagent

meta-externalagent: what it is and how to block or charge it

meta-externalagent is run by Meta. Meta's crawler for uses such as training AI models and improving its products.

Collects content for AI training
Run by
Meta
Its name in your logs
meta-externalagent
Published addresses
None, so it can only be recognised by name. Which crawlers publish them
Same company
FacebookBot

Block meta-externalagent in robots.txt

User-agent: meta-externalagent
Disallow: /

robots.txt is a request, not a lock: well-behaved crawlers follow it. To block by user agent at the server, match meta-externalagent in the User-Agent header.

Or charge meta-externalagent per page

Instead of blocking it, answer meta-externalagent with HTTP 402 Payment Required and a price. Crawlers and agents that support x402 pay a fraction of a cent in USDC straight to your wallet and get the page; people and search engines browse free. Set it up free on pay-per-crawl in about five minutes.

Is meta-externalagent reading your site?

See the full list of AI crawlers. Facts here follow each operator's own documentation; tell us if one changes.