The crawler question for Claude is ClaudeBot (training) and Claude-User (user-initiated fetches). Blocking is a real option with a real cost, and the two directions are not symmetric. Claude answers from model weights unless web search is enabled for the conversation. In its default mode there is no live retrieval at all, so what it says about your brand reflects what was in its training data: which is why the fix here is slower and more structural than on the live-retrieval engines.
How Claude actually decides
Claude answers from model weights unless web search is enabled for the conversation. In its default mode there is no live retrieval at all, so what it says about your brand reflects what was in its training data: which is why the fix here is slower and more structural than on the live-retrieval engines.
Its citations skew toward well-established, text-dense sources that were prominent at training time. In practice that means the surfaces worth auditing first are documentation sites, long-form editorial, Wikipedia, technical references.
The method, in order
Know which agent does what: ClaudeBot (training) and Claude-User (user-initiated fetches). Training access and answer-time retrieval are usually separate controls, and blocking the wrong one achieves the opposite of what was intended.
Decide what you are protecting. Blocking reduces the chance of being quoted without attribution; it also reduces the chance of being recommended at all.
Consider the asymmetry. Being absent from Claude is easy to cause and slow to undo, especially here, where the index does not refresh quickly.
If you do block, block deliberately and document it, so a future traffic investigation does not spend a week rediscovering it.
Re-check after any robots.txt change that the agent you meant to allow is still allowed.
The honest caveat
A brand founded or renamed recently can be absent from Claude entirely while doing well everywhere else, and no amount of on-page work fixes that quickly. It is the clearest example of why a single-engine tool misleads.
Cituna checks Claude daily alongside the other five engines, ChatGPT, Perplexity, Gemini, Claude, Grok and Google AI Overviews, on the $39 Starter, and pairs the result with Google Search Console so a change here can be tied to real clicks. We do not track Microsoft Copilot.
Drafted with AI assistance from our own research and Search Console data, and reviewed by Rahul A before publishing. Rules and prices change; check the linked official source before you act.