reverse DNS
Verifying AI bots by IP range and reverse DNS: a 2026 reference
Verify AI bots by IP range and reverse DNS: the official IP lists and DNS checks for OpenAI, Anthropic, Google, Perplexity, Bing, Apple, Amazon and more.
AI training crawler
Meta-ExternalAgent is a Meta crawler. Meta says it crawls the web for uses such as training foundation AI models or improving products by indexing content directly.
Meta instructs site owners to add a robots.txt disallow for the relevant crawler to block it, with no caveat for this one.
User-agent: meta-externalagent Allow: /
Meta does not publish a way to verify Meta-ExternalAgent. Treat the user agent alone as a claim, not proof.
Sill Agent Intelligence
reverse DNS
Verify AI bots by IP range and reverse DNS: the official IP lists and DNS checks for OpenAI, Anthropic, Google, Perplexity, Bing, Apple, Amazon and more.
Meta's AI crawlers
Meta's AI crawlers explained: what Meta-ExternalAgent, Meta-ExternalFetcher and Meta-WebIndexer do, how robots.txt applies, and which ones to allow.
Block it if you do not want your content used to train Meta's AI models or indexed by Meta. Meta says a robots.txt disallow is how to do it.
Meta's crawler page does not publish IP ranges or a verification method for it, so we cannot point you to an official check.