← All posts

Where does ChatGPT actually read from when it answers?

BrandLoop· GEO· AI search· technical SEO

Measurements from 2026 show ChatGPT’s search layer drawing on two sources: OpenAI’s own search index and scraped Google results. Overlap with Bing results has fallen to coincidence level. The first requirement for appearing in answers is that OAI-SearchBot can crawl the site. BrandLoop checks the same threshold on the user’s own page.

Anyone who wants to appear in AI answers has to start with one question: where does this system get its text? The answer changed over the last twelve months, and most published guidance still repeats the old one.

Four bots, four jobs

OpenAI documents four separate crawlers. Confusing them is the most common mistake site owners make.

OpenAI crawlers
OAI-SearchBotSearch. Appearing as a source in answers depends on this one.
GPTBotTraining foundation models.
ChatGPT-UserUser-initiated fetches of individual pages.
OAI-AdsBotSafety checks on pages submitted as ads in ChatGPT.

The distinction that matters: blocking GPTBot means opting out of training, not out of visibility. What determines whether you appear as a source is OAI-SearchBot, and OpenAI recommends allowing it explicitly.

“we recommend allowing OAI-SearchBot in your site’s robots.txt file”
— OpenAI — Bots documentation

One nuance is regularly missed: for user-initiated fetches, OpenAI states robots.txt rules may not apply. robots.txt is a permission mechanism, not a guarantee.

The Bing dependency is effectively over

Three independent lines point the same way. OpenAI does not name Bing in any current documentation, saying only that it sometimes partners with third-party search providers. Microsoft retired the Bing Search APIs entirely in August 2025. And a 2026 measurement across 1,249 ChatGPT answers and roughly 88,000 search results found ChatGPT using its own index plus scraped Google results, with only about 1.5% of URLs also appearing in Bing’s top 20.

To be precise: OpenAI never announced a separation. The measurement shows the dependency is gone in practice; no official statement confirms it. Those are different claims and worth keeping apart.

So is submitting a sitemap to Bing pointless?

No — the reason has changed. Bing’s index is no longer the main lever for ChatGPT, but it is what feeds Microsoft Copilot and Bing’s own AI summaries. Bing Webmaster Tools opened an AI Performance panel in February 2026 and states its scope plainly: Copilot, AI-generated summaries in Bing, and select partner integrations. Whether ChatGPT citations are inside that scope has not been disclosed by Microsoft.

The technical blocker people miss: the CDN

OpenAI’s help page sets two conditions: allow OAI-SearchBot to crawl, and confirm that your host or content delivery network permits traffic from OpenAI’s published crawler IP addresses. The second causes more real-world failures than the first, because AI-crawler blocking is on by default in some CDN plans. Your robots.txt can be perfect while the firewall turns the bot away at the door.

How citations are distributed

Measurements agree on two things. The funnel is narrow: in one study only 15% of pages ChatGPT retrieved made it into the final answer. And citations concentrate — an academic study measured the top 20 sources accounting for 67% of all citations in OpenAI models. Appearing in answers is a narrow race, not a broad one.

Classic search ranking remains a strong signal without being decisive. Across 15,000 prompts, 55.8% of cited pages sat in Google’s top 20, and pages ranking first were cited about 3.5 times more often than pages outside the top 20.

What measurably works, and what does not

  • llms.txt: no measured evidence in its favour. The only authoritative comment came from Google and was negative, and no AI vendor mentions the file in its crawler documentation.
  • schema.org markup: in a controlled study of 1,885 pages that already received citations, the effect was indistinguishable from zero. The study did not test discovery, so the honest reading is “no additional benefit measured on already-visible pages”, not “useless”.
  • Page size: one measurement found pages beyond 4 MB were not read at all.
  • Structured writing: pages carrying definitions, numeric data, comparisons and step-by-step procedures show measurably higher citation influence. That is the text’s own structure, not JSON-LD.

What this means for a personal brand

Whether a person appears when someone asks about them depends far more on what others have written than on edits to their own page — covered separately here. What this piece adds is the technical floor: if you have no crawlable page, or OAI-SearchBot cannot get through the door, none of the rest applies.

A five-minute checklist for your site

  • Open your robots.txt and confirm explicit allow lines for OAI-SearchBot, PerplexityBot, ClaudeBot and Google-Extended. A blanket allow may suffice, but naming them removes ambiguity about intent.
  • Check the AI-crawler blocking setting in your CDN dashboard. It ships on by default in some plans and operates independently of robots.txt.
  • View the page source of your homepage. If the text is not there — rendered only by JavaScript — some crawlers will not read it.
  • Check page weight. Pages bloated by large images and embedded video can be left unread.
  • Submit your sitemap to both Google Search Console and Bing Webmaster Tools. Bing is no longer the main lever for ChatGPT but it is what Copilot uses.

The whole list is about an hour of work, done once. Finishing it clears the technical floor — which is the precondition for visibility, not visibility itself. What decides the outcome is what others have written about you, and there is no shortcut for that.

Questions

Does ChatGPT still use Bing?
Measurements indicate the dependency is effectively gone. A 2026 study found ChatGPT using its own search index and scraped Google results, with roughly 1.5% overlap with Bing’s top 20. OpenAI has made no official statement about a separation.
Which bot do I need to allow to appear in ChatGPT answers?
OAI-SearchBot. OpenAI recommends allowing it explicitly. GPTBot is for training; blocking it does not directly remove you from search answers.
I allowed it in robots.txt but still don’t appear. Why?
Most often the CDN or firewall. OpenAI separately requires that your host accepts traffic from its published crawler IP addresses, and AI-crawler blocking ships on by default in some CDN plans.
Should I add an llms.txt file?
There is no measured evidence it helps. Google’s John Mueller stated no AI system uses it, and no vendor mentions it in crawler documentation. It has not been shown harmful either — its benefit is simply unmeasured.
Does schema.org markup increase AI citations?
In a controlled study of 1,885 pages that already received citations, the increase was indistinguishable from zero. The study did not test whether markup helps an unseen page get discovered. Treat it as hygiene, not leverage.

Sources

  1. OpenAI — Bots (OAI-SearchBot, GPTBot, ChatGPT-User, OAI-AdsBot)
  2. OpenAI — ChatGPT search help page
  3. Microsoft — Bing Search API retirement (11 August 2025)
  4. Bing Webmaster Blog — AI Performance (February 2026)
  5. Search Engine Land — measurement of ChatGPT’s retrieval stack
  6. Ahrefs — We tracked 1,885 pages adding schema
  7. News Source Citing Patterns in AI Search Systems (arXiv:2507.05301)