--- title: "GPTBot vs OAI-SearchBot vs ChatGPT-User | robots.txt | Len P. van der Hof" description: "Three OpenAI user-agents, three controls. GPTBot is the training-use signal. OAI-SearchBot decides eligibility for ChatGPT search answers. ChatGPT-User fetches on a person's request and is not the Search opt-out." image: "https://lenvanderhof.com/media/generated/blog-hero-gptbot-vs-oai-searchbot-v1.ce33aee4f410.wide.webp" --- [SEO, GEO and AEO](https://lenvanderhof.com/en/blog/category/seo-geo-aeo/) Research guide # GPTBot vs OAI-SearchBot vs ChatGPT-User: three bots, three jobs One vendor name hides three decisions: training, Search, and a fetch that a person started. Len P. van der HofPublished 21 September 20264 min read Training, Search, and a person-triggered fetch each use their own door. Direct answer GPTBot, OAI-SearchBot, and ChatGPT-User do different jobs for OpenAI. GPTBot crawls content that may be used to train OpenAI's foundation models; disallowing it signals that your content should not be used for that training. OAI-SearchBot surfaces websites in ChatGPT's search features; disallowing it keeps your content out of ChatGPT search answers, although a navigational link can still appear. ChatGPT-User fetches a page when a person asks ChatGPT or a Custom GPT to look at it. OpenAI says robots.txt rules may not apply to those user-initiated actions and that this agent is not used to decide Search inclusion. The GPTBot and OAI-SearchBot settings are independent, and OpenAI says its Search systems can take about 24 hours to reflect a robots.txt change. ## Key takeaways - GPTBot governs a signal about training use. OAI-SearchBot governs inclusion in ChatGPT search answers. They are separate switches. - ChatGPT-User is a person-triggered fetch. A log line from it proves nothing about Search indexing, and blocking it is not the Search opt-out. - OpenAI says the two crawl settings are independent and that one crawl may serve both purposes when both are allowed. - A correct robots.txt sets eligibility. It does not buy a citation, and it does not undo training that already happened. Blocking “the OpenAI bot” is not a policy. It is an unresolved choice hidden inside a singular noun. **Three OpenAI user-agents sit behind that noun, and each answers a different question.** `GPTBot` crawls content that may be used to train OpenAI’s foundation models; disallowing it tells OpenAI your content should not be used for that training. `OAI-SearchBot` decides whether your pages can surface in ChatGPT search answers. `ChatGPT-User` fetches a page because a person asked ChatGPT or a Custom GPT to look at it. The names rhyme. The controls do not. If you want ChatGPT Search to find your public pages while keeping them out of future training, you need two separate instructions. If you want the reverse, write the reverse pair. A rule for ChatGPT-User settles neither question. ## The comparison User-agentWhat starts the visitPublished purposeWhat a robots.txt disallow doesGPTBotAutomatic crawlingContent that may be used to make OpenAI’s generative AI foundation models more useful and safeSignals that your content should not be used in training those modelsOAI-SearchBotAutomatic crawling for SearchSurfacing websites in ChatGPT’s search featuresKeeps your content out of ChatGPT search answers; a navigational link may still appearChatGPT-UserA person’s request in ChatGPT or a Custom GPTVisiting a page to fulfil that requestLittle, on OpenAI’s own account: robots.txt rules may not apply, and this agent plays no part in Search inclusion OpenAI says the GPTBot and OAI-SearchBot settings are independent. It also says that when both are allowed, it may use one crawl for both purposes. Shared fetching does not merge the permissions. You still decide twice. The same crawler page lists a fourth agent, `OAI-AdsBot`, which validates landing pages submitted as ChatGPT ads. That is a separate job, and it stays outside this comparison. ## Two robots.txt policies Search on, training off: ``` User-agent: GPTBot Disallow: / User-agent: OAI-SearchBot Allow: / ``` Training on, Search off: ``` User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Disallow: / ``` Neither block is safe to paste blind. Under the robots.txt standard, a crawler follows the group that names it most specifically and falls back to `User-agent: *` only when no group does. A path rule, a second matching group, a CDN bot setting, or a firewall rule can change the effective answer. Test representative URLs against the whole file, not against the stanza you added. OpenAI publishes IP ranges for each agent. Use them when you need to tell a declared user-agent from an impersonator. There is no Search opt-out hidden inside ChatGPT-User. OpenAI assigns that job to OAI-SearchBot. If a page must stay unreachable even when a person asks a tool to fetch it, put it behind access control. robots.txt is an instruction to crawlers, not a lock. ## What each change does not do Disallowing GPTBot does not remove a page from ChatGPT Search. It also does not establish that anything fetched before the change has left a trained model. The control is a forward signal about training use. It is not a retroactive eraser. Disallowing OAI-SearchBot keeps your content out of ChatGPT search answers, on OpenAI’s stated policy. A title and a link can still surface as navigation. If that residue matters to you, read the current publisher guidance before treating robots.txt as a removal mechanism. A ChatGPT-User line in your access log proves one thing: that user-agent requested that URL. It does not prove that OAI-SearchBot indexed the page, that the page ranked, or that an answer cited it. Then let the change settle. OpenAI says its Search systems can take about 24 hours to adjust after a robots.txt update. A test ten minutes later measures your patience, not your policy. ## The operator check Write three answers before you open the file: 1. May this public content be considered for foundation-model training? 2. May it be surfaced inside ChatGPT search answers? 3. Does the page need real access control when a person asks a tool to fetch it? The first answer maps to GPTBot, the second to OAI-SearchBot, the third to your authentication design. Then test representative URLs against the complete robots.txt, and read the access log a day later. None of this buys a citation. The [citation-readiness audit](https://lenvanderhof.com/en/blog/citation-readiness-audit/) tests whether one eligible page can be extracted and checked. [How to get cited by ChatGPT](https://lenvanderhof.com/en/blog/how-to-get-cited-2026/) covers the wider source work. If the door is open and the name still does not appear, [Why am I not mentioned in ChatGPT?](https://lenvanderhof.com/en/blog/why-chatgpt-does-not-name-you/) is the diagnosis. [The Findability Engine](https://lenvanderhof.com/books/the-findability-engine/) owns the longer treatment; the book is in production and not for sale. The crawler policy only decides which door stands open. ## Terminology - [CITATION](https://lenvanderhof.com/glossary/citation/) - [AEO](https://lenvanderhof.com/glossary/aeo/) ## Sources 1. [Citation-readiness audit](https://lenvanderhof.com/en/blog/citation-readiness-audit/) 2. [How to get cited by ChatGPT](https://lenvanderhof.com/en/blog/how-to-get-cited-2026/) 3. [Why am I not mentioned in ChatGPT?](https://lenvanderhof.com/en/blog/why-chatgpt-does-not-name-you/) 4. [The Findability Engine](https://lenvanderhof.com/books/the-findability-engine/) 5. [Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots) · OpenAI 6. [Publishers and developers FAQ](https://help.openai.com/en/articles/12627856-publishers-and-developers-faq) · OpenAI Help Center ## Further reading - [Citation-readiness audit](https://lenvanderhof.com/en/blog/citation-readiness-audit/) - [How to get cited by ChatGPT](https://lenvanderhof.com/en/blog/how-to-get-cited-2026/) - [Why am I not mentioned in ChatGPT?](https://lenvanderhof.com/en/blog/why-chatgpt-does-not-name-you/) - [The Findability Engine](https://lenvanderhof.com/books/the-findability-engine/) About the author ## [Len P. van der Hof](https://lenvanderhof.com/en/authors/len-p-van-der-hof/) Entrepreneur, AI Innovator and Venture Builder Len P. van der Hof builds practical AI systems, digital ventures and evidence-informed tools for founders. ```json { "@context": "https://schema.org", "@graph": [ { "@type": "Person", "@id": "https://lenvanderhof.com/#person", "name": "Len P. van der Hof", "alternateName": [ "Len van der Hof", "L.P. van der Hof", "Leendert Pieter van der Hof" ], "honorificSuffix": "MSc", "url": "https://lenvanderhof.com/", "image": [ "https://lenvanderhof.com/photos/len-portrait-1.jpg", "https://lenvanderhof.com/photos/len-portrait-2.jpg", "https://lenvanderhof.com/photos/len-portrait-3.jpg", "https://lenvanderhof.com/photos/len-portrait-4.jpg", "https://lenvanderhof.com/photos/len-speaking.jpg", "https://lenvanderhof.com/photos/len-hero.jpg" ], "jobTitle": "Entrepreneur, AI Innovator and Venture Builder", "description": "Len P. van der Hof is a Dutch entrepreneur and AI innovator building ReasonKit, MindSesh, Undominated.ai, the fiction imprint LPH98.lifestyle, and technology ventures through LPH98.ventures.", "address": { "@type": "PostalAddress", "addressLocality": "Zwijndrecht", "addressCountry": "NL" }, "alumniOf": { "@type": "CollegeOrUniversity", "name": "Rotterdam School of Management, Erasmus University" }, "knowsAbout": [ "Artificial intelligence", "AI agents", "Agentic AI systems", "LLM routing", "SEO", "Generative engine optimization", "Answer engine optimization", "Venture building", "Founder performance", "Founder psychology", "Evidence-based decision-making" ], "sameAs": [ "https://www.linkedin.com/in/lenvanderhof/", "https://x.com/LenvanderHof", "https://www.youtube.com/channel/UCTG20buKqYYbitqqf7l3zJA", "https://www.instagram.com/Lenvanderhof/", "https://www.threads.com/@lenvanderhof", "https://github.com/Lenvanderhof", "https://huggingface.co/LPH98", "https://www.npmjs.com/~lenvanderhof", "https://www.goodreads.com/author/show/70983905.Len_P_van_der_Hof", "https://www.amazon.com/author/lenvanderhof", "https://www.bol.com/nl/nl/b/len-p-van-der-hof-msc/609879394/", "https://bsky.app/profile/lenvanderhof.com", "https://mastodon.social/@Lenvanderhof", "https://crates.io/users/Lenvanderhof", "https://cursor.com/@Lenvanderhof", "https://medium.com/@Lenvanderhof", "https://gitlab.com/Lenvanderhof", "https://hub.docker.com/u/lenvanderhof/", "https://dev.to/lenvanderhof", "https://www.facebook.com/Lenvanderhof", "https://soundcloud.com/Lenvanderhof" ], "affiliation": [ { "@id": "https://lenvanderhof.com/#publisher" }, { "@id": "https://lenvanderhof.com/#mindsesh" }, { "@id": "https://lenvanderhof.com/#lifestyle" } ] }, { "@type": "WebSite", "@id": "https://lenvanderhof.com/#website", "url": "https://lenvanderhof.com/", "name": "Len P. van der Hof", "description": "Len P. van der Hof is a Dutch entrepreneur and AI innovator building ReasonKit, MindSesh, Undominated.ai, the fiction imprint LPH98.lifestyle, and technology ventures through LPH98.ventures.", "inLanguage": [ "en", "nl" ], "publisher": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#publisher", "name": "LPH98.ventures", "url": "https://lph98.ventures", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#mindsesh", "name": "MindSesh", "url": "https://mindsesh.net", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "SoftwareApplication", "@id": "https://lenvanderhof.com/#reasonkit", "name": "ReasonKit", "url": "https://reasonkit.sh", "creator": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "SoftwareApplication", "@id": "https://lenvanderhof.com/#undominated", "name": "Undominated.ai", "url": "https://undominated.ai", "creator": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#lifestyle", "name": "LPH98.lifestyle", "url": "https://lph98.lifestyle", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "ImageObject", "@id": "https://lenvanderhof.com/en/blog/gptbot-vs-oai-searchbot/#primaryimage", "url": "https://lenvanderhof.com/media/generated/blog-hero-gptbot-vs-oai-searchbot-v1.ce33aee4f410.wide.webp", "contentUrl": "https://lenvanderhof.com/media/generated/blog-hero-gptbot-vs-oai-searchbot-v1.ce33aee4f410.wide.webp", "representativeOfPage": true }, { "@type": "BreadcrumbList", "@id": "https://lenvanderhof.com/en/blog/gptbot-vs-oai-searchbot/#breadcrumb", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://lenvanderhof.com/" }, { "@type": "ListItem", "position": 2, "name": "Blog", "item": "https://lenvanderhof.com/en/blog/" }, { "@type": "ListItem", "position": 3, "name": "SEO, GEO and AEO", "item": "https://lenvanderhof.com/en/blog/category/seo-geo-aeo/" }, { "@type": "ListItem", "position": 4, "name": "GPTBot vs OAI-SearchBot vs ChatGPT-User: three bots, three jobs", "item": "https://lenvanderhof.com/en/blog/gptbot-vs-oai-searchbot/" } ] }, { "@type": "WebPage", "@id": "https://lenvanderhof.com/en/blog/gptbot-vs-oai-searchbot/#webpage", "url": "https://lenvanderhof.com/en/blog/gptbot-vs-oai-searchbot/", "name": "GPTBot vs OAI-SearchBot vs ChatGPT-User: three bots, three jobs", "description": "Three OpenAI user-agents, three controls. GPTBot is the training-use signal. OAI-SearchBot decides eligibility for ChatGPT search answers. ChatGPT-User fetches on a person's request and is not the Search opt-out.", "isPartOf": { "@id": "https://lenvanderhof.com/#website" }, "primaryImageOfPage": { "@id": "https://lenvanderhof.com/en/blog/gptbot-vs-oai-searchbot/#primaryimage" }, "breadcrumb": { "@id": "https://lenvanderhof.com/en/blog/gptbot-vs-oai-searchbot/#breadcrumb" }, "inLanguage": "en-GB" }, { "@type": "BlogPosting", "@id": "https://lenvanderhof.com/en/blog/gptbot-vs-oai-searchbot/#article", "mainEntityOfPage": { "@id": "https://lenvanderhof.com/en/blog/gptbot-vs-oai-searchbot/#webpage" }, "headline": "GPTBot vs OAI-SearchBot vs ChatGPT-User: three bots, three jobs", "description": "Three OpenAI user-agents, three controls. GPTBot is the training-use signal. OAI-SearchBot decides eligibility for ChatGPT search answers. ChatGPT-User fetches on a person's request and is not the Search opt-out.", "datePublished": "2026-09-21T12:00:00.000Z", "author": { "@id": "https://lenvanderhof.com/#person" }, "publisher": { "@id": "https://lenvanderhof.com/#person" }, "image": [ "https://lenvanderhof.com/media/generated/blog-hero-gptbot-vs-oai-searchbot-v1.ce33aee4f410.square.webp", "https://lenvanderhof.com/media/generated/blog-hero-gptbot-vs-oai-searchbot-v1.ce33aee4f410.landscape.webp", "https://lenvanderhof.com/media/generated/blog-hero-gptbot-vs-oai-searchbot-v1.ce33aee4f410.wide.webp" ], "articleSection": "SEO, GEO and AEO", "inLanguage": "en-GB" } ] } ```