Who runs it
GPTBot is run by OpenAI, according to OpenAI, Overview of OpenAI Crawlers.
What OpenAI says it is for
OpenAI describes GPTBot this way.
“GPTBot is used to make our generative AI foundation models more useful and safe. It is used to crawl content that may be used in training our generative AI foundation models. Disallowing GPTBot indicates a site’s content should not be used in training generative AI foundation models.” OpenAI, Overview of OpenAI Crawlers
User-agent string and robots.txt token
Example user-agent string, from OpenAI (it says the version number may change).
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbotFrom OpenAI, Overview of OpenAI Crawlers.
robots.txt token: GPTBot
OpenAI says its robots.txt tags cover both GPTBot and OAI-SearchBot.
“OpenAI uses OAI-SearchBot and GPTBot robots.txt tags to enable webmasters to manage how their sites and content work with AI.” OpenAI, Overview of OpenAI Crawlers
Example rule (standard robots.txt syntax, written by us, not a vendor quote):
User-agent: GPTBot
Disallow: /
Does blocking it affect AI search visibility?
OpenAI says the settings are independent: a site can allow OAI-SearchBot to appear in search results while disallowing GPTBot. So, on OpenAI's own account, blocking GPTBot is a training choice, not a search choice.
OpenAI says GPTBot and OAI-SearchBot are set independently, which bears on search visibility.
“a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot to indicate that crawled content should not be used for training OpenAI’s generative AI foundation models.” OpenAI, Overview of OpenAI Crawlers
How to find it in your logs
For request-level inspection, use your host, CDN or web-server logs, as in SearchIntel’s log study. Search the user-agent field for the name. On a server with shell access:
grep -i "GPTBot" access.log
Google warns that user-agent strings can be spoofed, so verify the request before counting it.
OpenAI publishes the IP addresses GPTBot uses.
“Published IP addresses: https://openai.com/gptbot.json” OpenAI, Overview of OpenAI Crawlers
What we saw in our own logs
These figures come from our post How to Track AI Crawlers on Your Website: A Week of Logs: one week of logs (19 to 25 October 2026) from two websites, a specialist research site and a smaller services site.
- SearchIntel’s October log study reports for research site, observed requests and destination: “OpenAI's training crawler, GPTBot, made a handful of requests, all to the sitemap.”.
- SearchIntel’s October log study reports for services site, share of crawler requests: “GPTBot under 1%”.
Read this as a snapshot: the post says to “treat the shares as a snapshot, not a benchmark or a trend”, and says “We used the bot labels Vercel exported”, so the labels are Vercel’s. Full method and limits are in the post.
Frequently asked questions
What is GPTBot?
OpenAI says GPTBot crawls content that may be used in training its generative AI foundation models. Disallowing it tells OpenAI the site's content should not be used for that training.
Does blocking GPTBot affect AI search visibility?
OpenAI says the settings are independent: a site can allow OAI-SearchBot to appear in search results while disallowing GPTBot. So, on OpenAI's own account, blocking GPTBot is a training choice, not a search choice.
How do I find GPTBot in my logs?
Search the user-agent field of your host, CDN or web-server logs for the name, for example with grep -i "GPTBot" access.log. Google warns that user-agent strings can be spoofed, so verify the request before counting it.
Changelog
Last updated 17 November 2026.
- 17 November 2026: Pages published.
All bots: what each AI bot does on your website.