Definition
A user-triggered fetcher is an automated client that loads a web page because a person asked a product to, not as part of a crawl. Examples include ChatGPT-User, Perplexity-User and Google-Agent. Because a person requested the visit, operators such as Google and Perplexity say these fetchers generally ignore robots.txt rules.
Why it matters
User-triggered fetchers can be some of the most useful AI visits a store gets. Each one starts with a person asking an AI tool about your products or pages. A crawler collects pages in bulk; a fetcher loads one page because someone asked a question.
They also behave differently from crawlers under robots.txt. If you block a crawler such as GPTBot, that rule does not necessarily stop ChatGPT-User when a person asks ChatGPT to read your page. Knowing which kind of bot you are looking at helps you set the right rules.
How it works
Google defines the category: "User-triggered fetchers are initiated by users to perform a fetching function within a Google product." It adds: "Because the fetch was requested by a user, these fetchers generally ignore robots.txt rules." Google's list includes Google-Agent, which it says "is used by agents hosted on Google infrastructure to navigate the web and perform actions upon user request."
Other operators describe their fetchers in similar terms:
- OpenAI says ChatGPT-User may visit a page when a user asks ChatGPT a question, that it "is not used for crawling the web in an automatic fashion", and adds: "Because these actions are initiated by a user, robots.txt rules may not apply."
- Perplexity says Perplexity-User supports user actions in Perplexity, and: "Since a user requested the fetch, this fetcher generally ignores robots.txt rules."
Operators also publish ways to check these fetchers. Google and OpenAI each publish IP address lists for them, and Google says it is experimenting with Web Bot Auth using the https://agent.bot.goog identity.
Example
A shopper asks ChatGPT, "What's the return policy at this store?" ChatGPT loads your returns page once, with a user agent naming ChatGPT-User, and answers the question. Your robots.txt blocks GPTBot, but that rule names a different bot, and OpenAI says robots.txt rules may not apply to visits a user starts. To confirm it is genuine, match the IP address against OpenAI's published list. See the profiles for ChatGPT-User, Perplexity-User and Google-Agent, and robots.txt for AI agents.
Sources
- Google: User-triggered fetchers · checked 16 Nov 2026
- OpenAI: Overview of OpenAI crawlers · checked 16 Nov 2026
- Perplexity: Perplexity crawlers · checked 16 Nov 2026