AEO Glossary
ClaudeBot is Anthropic's official web crawler, identified by user-agent: Claude-Web or ClaudeBot. It crawls public websites to retrieve information used in Claude's responses to user queries. Like GPTBot, it respects robots.txt directives and operator-level content policies.
Claude is used by millions of users for research, writing, and analysis - and by enterprise teams via the Claude API. When users ask Claude about your industry, products, or brand, Claude retrieves information from its training data and, in some configurations, live web sources. Sites that allow ClaudeBot access have better representation in Claude's outputs.
Add User-agent: ClaudeBot / Allow: / and User-agent: Claude-Web / Allow: / to your robots.txt. Anthropic's usage policies and crawler information are documented at anthropic.com/aup.
See how your site scores on ClaudeBot
Run a free AEO audit and get a 6-category breakdown including this metric.
Run Free AEO Audit →