Skip to main content
Cituna
AI Visibility

Does blocking GPTBot remove me from ChatGPT?

Not by itself, and this is the most consequential robots.txt confusion in the category. GPTBot governs crawling for model training. OAI-SearchBot governs retrieval for ChatGPT's search feature.…

By Rahul AUpdated September 4, 20264 min read

See which of these you are already failing.

On this page
  1. Why this happens
  2. What to do about it
  3. Where Cituna fits

Not by itself, and this is the most consequential robots.txt confusion in the category. GPTBot governs crawling for model training. OAI-SearchBot governs retrieval for ChatGPT's search feature. Blocking GPTBot stops your content being used to train future models but leaves you eligible to be retrieved and cited in answers today. Blocking OAI-SearchBot is what removes you from search-grounded answers. Many sites blocked GPTBot intending to protect their content, kept OAI-SearchBot open, and are still cited, while others blocked both and quietly disappeared from the engine without ever connecting the two events.

Why this happens

The two agents exist because the two activities have different bargains. Training use is a one-way transfer: your content improves a model and you get nothing back. Retrieval is an exchange: your content is used and you are named, often with a link. Publishers who reason about those separately usually end up allowing one and refusing the other, which is exactly what the split makes possible.

The confusion persists because the failure is silent. Nothing warns you, no traffic report shows it, and the symptom, absence from one engine while search traffic holds steady, looks like a content problem for months before anyone checks the file.

What to do about it

Read your robots.txt as the engine does: check GPTBot and OAI-SearchBot separately, and confirm which one your rules actually match.

Decide deliberately rather than by inheritance. Most commercial sites want retrieval allowed and can take a view on training; a blanket block is rarely what anyone chose on purpose.

Record the decision in a comment in the file, so the next traffic investigation does not spend a week rediscovering it.

Where Cituna fits

Cituna's audit checks the named AI agents, GPTBot and OAI-SearchBot, PerplexityBot, ClaudeBot and Claude-User, Google-Extended, against your live robots.txt and flags the ones you are blocking, intentionally or not. It is the cheapest finding in the product and the one that most often explains a brand missing from a single engine.

Drafted with AI assistance from our own research and Search Console data, and reviewed by Rahul A before publishing. Rules and prices change; check the linked official source before you act.

Frequently asked questions

Does blocking GPTBot remove me from ChatGPT?

Not by itself, and this is the most consequential robots.txt confusion in the category. GPTBot governs crawling for model training. OAI-SearchBot governs retrieval for ChatGPT's search feature. Blocking GPTBot stops your content being used to train future models but leaves you eligible to be retrieved and cited in answers today. Blocking OAI-SearchBot is what removes you from search-grounded answers. Many sites blocked GPTBot intending to protect their content, kept OAI-SearchBot open, and are still cited, while others blocked both and quietly disappeared from the engine without ever connecting the two events.

What about Claude and Perplexity, same split?

Similar in shape, different in naming. Anthropic operates ClaudeBot for training and Claude-User for user-initiated fetches; Perplexity operates PerplexityBot for its retrieval. The principle holds across all of them: check the specific named agents rather than assuming one rule covers a vendor.

Does blocking Google-Extended remove me from AI Overviews?

No, and this catches people out. Google-Extended controls whether your content is used for Gemini training. AI Overviews are generated from the main Search index via Googlebot, so blocking Google-Extended leaves your AI Overview presence intact. Removing yourself from AI Overviews requires different controls entirely.

See how AI engines see your brand

Start a free 3-day trial and see the exact buyer prompts you lose across ChatGPT, Perplexity, Gemini, Claude, Grok and Google AI Overviews, with a prioritized AEO, GEO and SEO action plan and the fixes to win them.

3-day free trial · Card required, cancel anytime · Works with ChatGPT, Perplexity, Gemini, Claude, Grok and Google AI Overviews

Start free trial