OpenAI's web crawler, used to gather content for training and to fetch pages when ChatGPT browses the web.
ID
SS-DF-107
Confidence
Established · 79
Evidence
Established
Updated
2026-07-08
Definition
OpenAI's web crawler, used to gather content for training and to fetch pages when ChatGPT browses the web.
GPTBot is the user agent OpenAI uses to crawl the web, and site owners can allow or block it through robots.txt. Blocking it can keep your content out of future training data and, depending on configuration, out of live retrieval, which reduces the chance ChatGPT can accurately represent or cite your brand. The decision is a trade-off between controlling how your content is used and staying visible in one of the largest AI answer channels.