--- title: "ChatGPT, Perplexity, AI Overviews: three doors to your page | Len P. van der Hof" description: ChatGPT search, Perplexity and Google AI Overviews reach your page through different crawlers and switches. A table from official docs and a 15-minute check. image: "https://lenvanderhof.com/media/generated/blog-hero-three-answer-engines-three-doors-v1.d824f6b59f31.wide.webp" --- [SEO, GEO and AEO](https://lenvanderhof.com/en/blog/category/seo-geo-aeo/) Research guide # ChatGPT, Perplexity, Google AI Overviews: three engines, three doors to your page Opening one door does nothing for the other two. Len P. van der HofPublished 8 October 20267 min read Three doors, three keys. One of them is open. Direct answer ChatGPT search, Perplexity and Google's AI Overviews each reach your page through a different door. ChatGPT search needs OpenAI's OAI-SearchBot crawler to be allowed. Perplexity needs PerplexityBot. Google's AI Overviews and AI Mode draw on the ordinary Google Search index, so they need Googlebot, indexing and snippet eligibility, with Search Console's generative AI setting left on include. Opening one door does nothing for the others, and none of the three publishes how it picks the sources it cites. ## Key takeaways - ChatGPT search is controlled by OAI-SearchBot, Perplexity by PerplexityBot, and Google's AI Overviews and AI Mode by Googlebot plus Google's snippet rules. - Training switches are separate doors. GPTBot and Google-Extended do not decide whether you appear in ChatGPT search or in Google Search. - The fetchers that act when a person asks play by other rules: OpenAI says robots.txt may not apply to ChatGPT-User, and Perplexity says Perplexity-User generally ignores it. - No vendor publishes a formula for choosing citations. OpenAI's own help page says placement is not guaranteed. **ChatGPT search, Perplexity and Google’s AI Overviews each reach your page through a different door, and each door has its own key.** ChatGPT search needs OpenAI’s search crawler, OAI-SearchBot, to be allowed. Perplexity needs its crawler, PerplexityBot. Google’s AI Overviews and AI Mode use the ordinary Google Search index, so they need Googlebot, a page that can be indexed, and permission to show a snippet. Opening one door does nothing for the other two. A few words first, because the rest depends on them. An answer engine is a service that writes an answer to a question and names its sources, instead of only showing a list of links. A crawler is the program a company sends to fetch web pages. robots.txt is the text file at the root of your site that tells crawlers, by name, what they may fetch. An index is the stored copy of pages that a search engine looks things up in. ## The three doors in one table Everything in this table comes from the vendors’ own documentation, read on 24 September 2026. ChatGPT searchPerplexityGoogle AI Overviews and AI ModeWhat feeds the answerPages crawled by OAI-SearchBot, plus other search providers OpenAI sometimes partners withPages crawled by PerplexityBotGoogle’s own Search indexrobots.txt name that controls itOAI-SearchBotPerplexityBotGooglebotOther switches you will meetGPTBot (training), ChatGPT-User (fetches when a person asks)Perplexity-User (fetches when a person asks)Snippet rules, the Search Console generative AI setting, Google-Extended (not a Search switch)What the docs say about linksAnswers “may include citations”; a citation opens its sourcePerplexityBot exists “to surface and link websites”Eligible pages can be shown “as a supporting link”How to check your accessLook for OAI-SearchBot in your server logs, from the IP ranges OpenAI publishesLook for PerplexityBot in your logs, from Perplexity’s published IP rangesURL Inspection in Search Console; the Generative AI performance reportHow long a change takesAbout 24 hours after a robots.txt changeUp to 24 hoursSeveral days to several months for snippet-control changes The last Google cell comes from its AI features page, which says recrawling “can take anywhere from several days to several months” after you change preview controls such as snippet rules. ## Door one: ChatGPT search OpenAI’s publishers FAQ starts from a generous premise: “Any public website can appear in ChatGPT search.” Its crawler page says OAI-SearchBot “is used to surface websites in search results in ChatGPT’s search features.” Sites that block it “will not be shown in ChatGPT search answers, though can still appear as navigational links.” OpenAI’s help page adds the second half of the key: allow the crawler and confirm that your host or content delivery network (CDN, the service many sites sit behind for speed and filtering) lets through OpenAI’s published searchbot IP addresses. Two details matter. First, OpenAI’s help page says “ChatGPT search sometimes partners with other search providers”, rewriting the question into targeted searches, and it links Microsoft’s and Shopify’s privacy policies in that section. Your own crawl access is one input, not the only one. Second, GPTBot is a different door: it collects content that may be used to train OpenAI’s models. Blocking it does not take you out of ChatGPT search. [GPTBot vs OAI-SearchBot vs ChatGPT-User](https://lenvanderhof.com/en/blog/gptbot-vs-oai-searchbot/) goes through all three OpenAI names. ## Door two: Perplexity Perplexity’s crawler page says PerplexityBot “is designed to surface and link websites in search results on Perplexity. It is not used to crawl content for AI foundation models.” It recommends allowing the bot in robots.txt and letting through requests from its published IP ranges, and it gives step-by-step firewall settings for Cloudflare and AWS. Its second agent, Perplexity-User, visits a page when a person asks a question and may “include a link to the page in its response.” Perplexity says that because a person requested the fetch, it “generally ignores robots.txt rules.” So a robots.txt block on Perplexity-User is not a reliable way to keep a page out. If a page must stay private, put it behind a login. ## Door three: Google AI Overviews and AI Mode Google has no separate AI crawler for these features. Its AI features page says “robots.txt directives for Googlebot is the control for site owners to manage access to how their sites are crawled for Search.” To be eligible as a supporting link, “a page must be indexed and eligible to be shown in Google Search with a snippet”, and Google adds that “There are no additional technical requirements.” Three extra switches sit around that door. The snippet rules `nosnippet`, `max-snippet` and `data-nosnippet`, plus the indexing rule `noindex`, limit what Google may show or use. Search Console has a generative AI setting where include is the default; Google says it had reached all websites by 31 August 2026. And Google-Extended, a robots.txt token for Gemini training and grounding, “does not impact a site’s inclusion in Google Search nor is it used as a ranking signal in Google Search.” Even with every switch right, you may see nothing. Google says AI Overviews “often don’t trigger”, because they only appear when its systems judge them useful on top of the ordinary results. ## What does no vendor tell you? How it chooses. OpenAI’s help page is the most direct: “ChatGPT ranks search results using multiple factors intended to help users find relevant, reliable information. Placement is not guaranteed.” Google says its AI features are rooted in its core Search ranking systems and that indexing and serving are not guaranteed. Perplexity’s crawler page says nothing about selection at all. So when someone offers you “the ChatGPT citation formula”, they are selling a guess. What you can control is covered in [How answer engines choose sources](https://lenvanderhof.com/en/blog/how-answer-engines-select-sources/), and the page work in [How to get cited by ChatGPT](https://lenvanderhof.com/en/blog/how-to-get-cited-2026/). Opening the doors is the part of [AEO](https://lenvanderhof.com/glossary/aeo/) (answer engine optimization) that you can check with certainty. ## Four myths and the facts MythFact, from the vendor’s own page”Blocking GPTBot keeps me out of ChatGPT.”GPTBot is the training crawler. ChatGPT search uses OAI-SearchBot.”Blocking Google-Extended keeps me out of AI Overviews.”Google says Google-Extended does not affect inclusion in Google Search. AI Overviews follow Googlebot, snippet rules and the Search Console setting.”A robots.txt line for Perplexity-User keeps Perplexity away.”Perplexity says Perplexity-User generally ignores robots.txt, because a person asked.”If I let all the bots in, I will be cited.”Access makes you eligible. OpenAI: “Placement is not guaranteed.” ## A worked example (hypothetical) Imagine a Dutch company that sells accounting software. Its hosting company switches on stricter bot protection one spring. Google traffic carries on as normal, and the page on invoicing rules still shows up in AI Overviews, because Googlebot is let through. But the server logs show no OAI-SearchBot or PerplexityBot visits for weeks, only blocked requests. robots.txt allows both bots. The firewall does not. The fix is a rule that lets through the IP ranges OpenAI and Perplexity publish. Nothing about that fix promises a citation. It only puts two of the three doors back on their hinges. ## The 15-minute access check 1. **Minutes 0 to 3.** Open `https://your-site/robots.txt`. Find the groups that name OAI-SearchBot, PerplexityBot and Googlebot, and the `User-agent: *` group. Write down open or closed for each. 2. **Minutes 3 to 6.** Open your firewall or CDN settings. Look for bot protection, challenge pages or country blocks that could stop the crawlers before they read robots.txt. 3. **Minutes 6 to 10.** Search the last seven days of server logs for `OAI-SearchBot`, `PerplexityBot` and `Googlebot`. Note the status codes: 200 means served, 403 means refused. Check a few addresses against the IP lists OpenAI and Perplexity publish, because anyone can fake a user-agent name. 4. **Minutes 10 to 13.** In Search Console, run URL Inspection on your most important page. Then open the generative AI setting and confirm it says include. 5. **Minutes 13 to 15.** Write three lines: ChatGPT door, Perplexity door, Google door. Each one is open, closed or unknown. If a line says unknown, that is your next job. The book that treats this as one system, [The Findability Engine](https://lenvanderhof.com/books/the-findability-engine/), is in production and not for sale yet. The check above needs no book: a text file, a firewall screen and a log. Cite this:ChatGPT, Perplexity, Google AI Overviews: three engines, three doors to your page.Len P. van der Hof. [https://lenvanderhof.com/en/blog/three-answer-engines-three-doors/](https://lenvanderhof.com/en/blog/three-answer-engines-three-doors/) · Published 8 October 2026. ## Sources 1. [Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots) · OpenAI 2. [Publishers and developers FAQ](https://help.openai.com/en/articles/12627856-publishers-and-developers-faq) · OpenAI Help Center 3. [Searching the web with ChatGPT](https://help.openai.com/en/articles/9237897-chatgpt-search) · OpenAI Help Center 4. [Perplexity Crawlers](https://docs.perplexity.ai/guides/bots) · Perplexity 5. [AI features and your website](https://developers.google.com/search/docs/appearance/ai-features) · Google Search Central 6. [Optimizing your website for generative AI features on Google Search](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) · Google Search Central 7. [Google's common crawlers (Google-Extended)](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers) · Google for Developers 8. [Search generative AI control](https://support.google.com/webmasters/answer/16908024?hl=en) · Google Search Console Help 9. [Generative AI performance report (Search)](https://support.google.com/webmasters/answer/16984139?hl=en) · Google Search Console Help 10. [GPTBot vs OAI-SearchBot vs ChatGPT-User](https://lenvanderhof.com/en/blog/gptbot-vs-oai-searchbot/) 11. [How answer engines choose sources](https://lenvanderhof.com/en/blog/how-answer-engines-select-sources/) 12. [How to get cited by ChatGPT](https://lenvanderhof.com/en/blog/how-to-get-cited-2026/) 13. [AEO (glossary)](https://lenvanderhof.com/glossary/aeo/) 14. [The Findability Engine](https://lenvanderhof.com/books/the-findability-engine/) ## Further reading - [GPTBot vs OAI-SearchBot vs ChatGPT-User](https://lenvanderhof.com/en/blog/gptbot-vs-oai-searchbot/) - [How answer engines choose sources](https://lenvanderhof.com/en/blog/how-answer-engines-select-sources/) - [How to get cited by ChatGPT](https://lenvanderhof.com/en/blog/how-to-get-cited-2026/) About the author ## [Len P. van der Hof](https://lenvanderhof.com/en/authors/len-p-van-der-hof/) Entrepreneur, AI Innovator and Venture Builder Len P. van der Hof builds practical AI systems, digital ventures and evidence-informed tools for founders. ```json { "@context": "https://schema.org", "@graph": [ { "@type": "Person", "@id": "https://lenvanderhof.com/#person", "name": "Len P. van der Hof", "alternateName": [ "Len van der Hof", "L.P. van der Hof", "Leendert Pieter van der Hof" ], "honorificSuffix": "MSc", "url": "https://lenvanderhof.com/", "image": [ "https://lenvanderhof.com/photos/len-portrait-1.jpg", "https://lenvanderhof.com/photos/len-portrait-2.jpg", "https://lenvanderhof.com/photos/len-portrait-3.jpg", "https://lenvanderhof.com/photos/len-portrait-4.jpg", "https://lenvanderhof.com/photos/len-speaking.jpg", "https://lenvanderhof.com/photos/len-hero.jpg" ], "jobTitle": "Entrepreneur, AI Innovator and Venture Builder", "description": "Len P. van der Hof, MSc, is a Dutch entrepreneur and AI innovator in Zwijndrecht. He builds ReasonKit, MindSesh, Undominated.ai, books under his name, the fiction imprint LPH98.lifestyle, and technology ventures through LPH98.ventures. Eleven titles in Systems for the Strategic Self are available now, in English and Dutch.", "address": { "@type": "PostalAddress", "addressLocality": "Zwijndrecht", "addressCountry": "NL" }, "alumniOf": { "@type": "CollegeOrUniversity", "name": "Rotterdam School of Management, Erasmus University" }, "knowsAbout": [ "Artificial intelligence", "AI agents", "Agentic AI systems", "LLM routing", "SEO", "Generative engine optimization", "Answer engine optimization", "Venture building", "Founder performance", "Founder psychology", "Evidence-based decision-making" ], "sameAs": [ "https://www.linkedin.com/in/lenvanderhof/", "https://x.com/LenvanderHof", "https://www.youtube.com/channel/UCTG20buKqYYbitqqf7l3zJA", "https://www.instagram.com/Lenvanderhof/", "https://www.threads.com/@lenvanderhof", "https://github.com/Lenvanderhof", "https://huggingface.co/LPH98", "https://www.npmjs.com/~lenvanderhof", "https://www.goodreads.com/author/show/70983905.Len_P_van_der_Hof", "https://www.amazon.com/author/lenvanderhof", "https://www.bol.com/nl/nl/b/len-p-van-der-hof-msc/609879394/", "https://bsky.app/profile/lenvanderhof.com", "https://mastodon.social/@Lenvanderhof", "https://crates.io/users/Lenvanderhof", "https://cursor.com/@Lenvanderhof", "https://medium.com/@Lenvanderhof", "https://gitlab.com/Lenvanderhof", "https://hub.docker.com/u/lenvanderhof/", "https://dev.to/lenvanderhof", "https://www.facebook.com/Lenvanderhof", "https://soundcloud.com/Lenvanderhof" ], "affiliation": [ { "@id": "https://lenvanderhof.com/#publisher" }, { "@id": "https://lenvanderhof.com/#mindsesh" }, { "@id": "https://lenvanderhof.com/#lifestyle" } ] }, { "@type": "WebSite", "@id": "https://lenvanderhof.com/#website", "url": "https://lenvanderhof.com/", "name": "Len P. van der Hof", "description": "Len P. van der Hof, MSc, is a Dutch entrepreneur and AI innovator in Zwijndrecht. He builds ReasonKit, MindSesh, Undominated.ai, books under his name, the fiction imprint LPH98.lifestyle, and technology ventures through LPH98.ventures. Eleven titles in Systems for the Strategic Self are available now, in English and Dutch.", "inLanguage": [ "en", "nl" ], "publisher": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#publisher", "name": "LPH98.ventures", "url": "https://lph98.ventures", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#mindsesh", "name": "MindSesh", "url": "https://mindsesh.net", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "SoftwareApplication", "@id": "https://lenvanderhof.com/#reasonkit", "name": "ReasonKit", "url": "https://reasonkit.sh", "creator": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "SoftwareApplication", "@id": "https://lenvanderhof.com/#undominated", "name": "Undominated.ai", "url": "https://undominated.ai", "creator": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#lifestyle", "name": "LPH98.lifestyle", "url": "https://lph98.lifestyle", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "ImageObject", "@id": "https://lenvanderhof.com/en/blog/three-answer-engines-three-doors/#primaryimage", "url": "https://lenvanderhof.com/media/generated/blog-hero-three-answer-engines-three-doors-v1.d824f6b59f31.wide.webp", "contentUrl": "https://lenvanderhof.com/media/generated/blog-hero-three-answer-engines-three-doors-v1.d824f6b59f31.wide.webp", "representativeOfPage": true }, { "@type": "BreadcrumbList", "@id": "https://lenvanderhof.com/en/blog/three-answer-engines-three-doors/#breadcrumb", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://lenvanderhof.com/" }, { "@type": "ListItem", "position": 2, "name": "Blog", "item": "https://lenvanderhof.com/en/blog/" }, { "@type": "ListItem", "position": 3, "name": "SEO, GEO and AEO", "item": "https://lenvanderhof.com/en/blog/category/seo-geo-aeo/" }, { "@type": "ListItem", "position": 4, "name": "ChatGPT, Perplexity, Google AI Overviews: three engines, three doors to your page", "item": "https://lenvanderhof.com/en/blog/three-answer-engines-three-doors/" } ] }, { "@type": "WebPage", "@id": "https://lenvanderhof.com/en/blog/three-answer-engines-three-doors/#webpage", "url": "https://lenvanderhof.com/en/blog/three-answer-engines-three-doors/", "name": "ChatGPT, Perplexity, Google AI Overviews: three engines, three doors to your page", "description": "ChatGPT search, Perplexity and Google AI Overviews reach your page through different crawlers and switches. A table from official docs and a 15-minute check.", "isPartOf": { "@id": "https://lenvanderhof.com/#website" }, "primaryImageOfPage": { "@id": "https://lenvanderhof.com/en/blog/three-answer-engines-three-doors/#primaryimage" }, "breadcrumb": { "@id": "https://lenvanderhof.com/en/blog/three-answer-engines-three-doors/#breadcrumb" }, "inLanguage": "en-GB" }, { "@type": "BlogPosting", "@id": "https://lenvanderhof.com/en/blog/three-answer-engines-three-doors/#article", "mainEntityOfPage": { "@id": "https://lenvanderhof.com/en/blog/three-answer-engines-three-doors/#webpage" }, "headline": "ChatGPT, Perplexity, Google AI Overviews: three engines, three doors to your page", "description": "ChatGPT search, Perplexity and Google AI Overviews reach your page through different crawlers and switches. A table from official docs and a 15-minute check.", "datePublished": "2026-10-08T10:17:00.000Z", "author": { "@id": "https://lenvanderhof.com/#person" }, "publisher": { "@id": "https://lenvanderhof.com/#person" }, "image": [ "https://lenvanderhof.com/media/generated/blog-hero-three-answer-engines-three-doors-v1.d824f6b59f31.square.webp", "https://lenvanderhof.com/media/generated/blog-hero-three-answer-engines-three-doors-v1.d824f6b59f31.landscape.webp", "https://lenvanderhof.com/media/generated/blog-hero-three-answer-engines-three-doors-v1.d824f6b59f31.wide.webp" ], "articleSection": "SEO, GEO and AEO", "inLanguage": "en-GB" } ] } ```