All AI bots
AI Bots

What GPTBot Does on Your Website

What OpenAI says it is for, how to spot it in your logs, and what we saw in ours.

In short: OpenAI says GPTBot crawls content that may be used in training its generative AI foundation models. Disallowing it tells OpenAI the site's content should not be used for that training.

Who runs it

GPTBot is run by OpenAI, according to OpenAI, Overview of OpenAI Crawlers.

What OpenAI says it is for

OpenAI describes GPTBot this way.

“GPTBot is used to make our generative AI foundation models more useful and safe. It is used to crawl content that may be used in training our generative AI foundation models. Disallowing GPTBot indicates a site’s content should not be used in training generative AI foundation models.” OpenAI, Overview of OpenAI Crawlers

User-agent string and robots.txt token

Example user-agent string, from OpenAI (it says the version number may change).

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot

From OpenAI, Overview of OpenAI Crawlers.

robots.txt token: GPTBot

OpenAI says its robots.txt tags cover both GPTBot and OAI-SearchBot.

“OpenAI uses OAI-SearchBot and GPTBot robots.txt tags to enable webmasters to manage how their sites and content work with AI.” OpenAI, Overview of OpenAI Crawlers

Example rule (standard robots.txt syntax, written by us, not a vendor quote):

User-agent: GPTBot
Disallow: /

Does blocking it affect AI search visibility?

OpenAI says the settings are independent: a site can allow OAI-SearchBot to appear in search results while disallowing GPTBot. So, on OpenAI's own account, blocking GPTBot is a training choice, not a search choice.

OpenAI says GPTBot and OAI-SearchBot are set independently, which bears on search visibility.

“a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot to indicate that crawled content should not be used for training OpenAI’s generative AI foundation models.” OpenAI, Overview of OpenAI Crawlers

How to find it in your logs

For request-level inspection, use your host, CDN or web-server logs, as in SearchIntel’s log study. Search the user-agent field for the name. On a server with shell access:

grep -i "GPTBot" access.log

Google warns that user-agent strings can be spoofed, so verify the request before counting it.

OpenAI publishes the IP addresses GPTBot uses.

“Published IP addresses: https://openai.com/gptbot.json” OpenAI, Overview of OpenAI Crawlers

What we saw in our own logs

These figures come from our post How to Track AI Crawlers on Your Website: A Week of Logs: one week of logs (19 to 25 October 2026) from two websites, a specialist research site and a smaller services site.

Read this as a snapshot: the post says to “treat the shares as a snapshot, not a benchmark or a trend”, and says “We used the bot labels Vercel exported”, so the labels are Vercel’s. Full method and limits are in the post.

Frequently asked questions

What is GPTBot?

OpenAI says GPTBot crawls content that may be used in training its generative AI foundation models. Disallowing it tells OpenAI the site's content should not be used for that training.

Does blocking GPTBot affect AI search visibility?

OpenAI says the settings are independent: a site can allow OAI-SearchBot to appear in search results while disallowing GPTBot. So, on OpenAI's own account, blocking GPTBot is a training choice, not a search choice.

How do I find GPTBot in my logs?

Search the user-agent field of your host, CDN or web-server logs for the name, for example with grep -i "GPTBot" access.log. Google warns that user-agent strings can be spoofed, so verify the request before counting it.

Changelog

Last updated 17 November 2026.

All bots: what each AI bot does on your website.

See Whether AI Recommends Your Brand

Run a free check across ChatGPT, Claude, Gemini, Perplexity and Google AI Mode. See your score, your platform breakdown, and the competitors AI recommends instead.

Related bots