AEO Glossary
GPTBot is OpenAI's official web crawler, identified in HTTP request headers as user-agent: GPTBot. It crawls publicly accessible websites to collect text used in training and improving ChatGPT and other OpenAI models. It respects robots.txt directives.
ChatGPT has over 200 million active users who ask questions about brands, products, and services daily. If GPTBot cannot crawl your site, ChatGPT may have outdated, incorrect, or no information about your brand. Allowing GPTBot access is the minimum requirement for AEO with the world's most-used AI assistant.
Add User-agent: GPTBot / Allow: / to your robots.txt. To block GPTBot from specific paths (e.g., private pricing pages), use Disallow: /private/. OpenAI's crawler documentation is at platform.openai.com/docs/gptbot.
See how your site scores on GPTBot
Run a free AEO audit and get a 6-category breakdown including this metric.
Run Free AEO Audit →