How ChatGPT search finds and cites sources, and what a site owner controls

The one control OpenAI documents for site owners is robots.txt: allow OAI-SearchBot and your pages are eligible to appear in ChatGPT search answers; block it and they are not shown, though they can still appear as navigational links. Blocking GPTBot, OpenAI's training crawler, does not remove you from search.

If you run marketing at a SaaS, fintech or tech company, or at any growing business that sells to other businesses, this matters because your buyers now ask ChatGPT about vendors before they book a call. Whether ChatGPT can read your pricing page, your security page and your comparison pages decides whether it describes you from your own words or from someone else's. This guide covers what OpenAI says, what has been reported but not confirmed, what you can control, and how to check it. It is one of our engine guides; the AI search visibility funnel explains how they fit together.

Which index does ChatGPT search retrieve from?

ChatGPT search retrieves from more than one index, and OpenAI has not said how they are weighted. What is on the record:

• OpenAI's own crawler. OpenAI documents OAI-SearchBot as the crawler that surfaces websites in ChatGPT's search features. That makes OpenAI's own crawl a confirmed part of the system.
• Bing. At launch in October 2024, OpenAI's search lead Adam Fry told The Verge that ChatGPT search was built with "a mix of search technologies," including Microsoft's Bing (The Verge, October 31, 2024). Microsoft's own Bing Webmaster blog says Bing powers "AI tools like ChatGPT" (Bing Webmaster Blog, June 2025).
• A first-party index, partly documented. OpenAI's web search guide lets a developer "Control whether the web search tool fetches live content or uses only cached/indexed results", and says to "Set external_web_access: false on the web_search tool to run in offline/cache-only mode." So OpenAI keeps a cached index of web content and can answer from it without a live fetch. The guide describes the API; it does not say how ChatGPT itself uses that store.
• The goal for that index. In sworn testimony at the Google search remedies trial in April 2025, ChatGPT's head of product Nick Turley said ChatGPT pulls from Bing's search information, and that OpenAI had hoped to serve 80% of ChatGPT's searches from its own index, a goal he described as years away (Courthouse News Service, April 2025). Accounts of the testimony differ on the original target date, so we do not quote one.

So the practical answer is: being crawlable by OAI-SearchBot is the documented requirement, and being indexed in Bing is a reasonable second step because Bing is a named part of the mix. Neither guarantees a citation. Getting into an index makes you eligible; whether you are read for a given question is a separate stage we cover in Ranking is not retrieval .

How does ChatGPT decide when to search the web vs use training data?

OpenAI's ChatGPT decides question by question whether to search the web or answer from its training data. The model makes the choice from what the prompt asks. OpenAI documents this for its developer API in its web search guide: "Like any other tool, the model can choose to search the web or not based on the content of the input prompt", and reasoning models can search several times and "decide whether to keep searching". OpenAI does not publish the rule the ChatGPT app itself uses, so read the API description as the closest documented guide, not as the app's exact behaviour.

Two consequences for a site owner:

• A question that needs current or specific facts gives the model a reason to search. Prices, recent changes and details about a named company are the kind of thing training data may not hold. A question it can answer from training may get no search at all, and then no citation.
• What it learned in training still frames the answer. When ChatGPT does search, the pages it retrieves sit next to what the model already believes about your category and your company, so an outdated description of you on other sites can survive a search.

A user can also ask it to search ("search the web for...") in the prompt itself.

Which crawler fetches for ChatGPT, and which crawler is for training?

OAI-SearchBot fetches for ChatGPT search, and GPTBot is the training crawler. OpenAI runs four user agents, and each one is controlled separately in robots.txt:

The distinction that trips people up: blocking GPTBot to keep your content out of training does nothing to your search visibility, and blocking OAI-SearchBot removes you from search answers even if GPTBot is allowed. One detail runs the other way: "If your site has allowed both bots, we may use the results from just one crawl for both use cases to avoid duplicative crawling." So a site that allows both may see fewer visits than it expects from each.

A robots.txt that keeps you in ChatGPT search while opting out of training:

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

Set it on every subdomain you care about. OpenAI also says that when its crawlers fetch robots.txt itself, they may add a robots.txt marker to the user-agent string, which helps you tell those requests apart in logs that do not record paths. We explain crawler access across all engines in Being indexable by AI search engines.

What does OpenAI itself document?

OpenAI documents its crawlers on one page, Overview of OpenAI Crawlers. We quote it from the live page as fetched on October 6, 2026.

On the search crawler:

"OAI-SearchBot is for search. OAI-SearchBot is used to surface websites in search results in ChatGPT’s search features. Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links."

On how to be eligible:

"To help ensure your site appears in search results, we recommend allowing OAI-SearchBot in your site’s robots.txt file and allowing requests from our published IP ranges below."

On search and training being separate:

"a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot to indicate that crawled content should not be used for training OpenAI’s generative AI foundation models."

On timing:

"For search results, please note it can take ~24 hours from a site’s robots.txt update for our systems to adjust."

On user-initiated visits:

"ChatGPT-User is not used for crawling the web in an automatic fashion. Because these actions are initiated by a user, robots.txt rules may not apply."

"ChatGPT-User is not used to determine whether content may appear in Search. Please use OAI-SearchBot in robots.txt for managing Search opt outs and automatic crawl."

OpenAI publishes the IP addresses OAI-SearchBot uses as a JSON file, so you can confirm a request claiming to be OAI-SearchBot really came from OpenAI.

OpenAI's web search guide describes how its developer API searches, which is useful for testing, with a caveat we explain in our measurement section below: the guide is about the API, not about the ChatGPT app.

What OpenAI does not document: a URL submission tool, a sitemap intake, or any webmaster console for ChatGPT.

What is reported about ChatGPT search but not confirmed?

Four claims about ChatGPT search are widely repeated but not confirmed by OpenAI. Each has a real source behind it.

1. How much of ChatGPT search comes from Bing today. The Verge quote and Nick Turley's testimony establish that Bing is part of the mix. Neither gives a current share. Claims that ChatGPT search "is Bing" go beyond the evidence.
2. That ChatGPT also uses Google results. The Information reported in August 2025 that OpenAI has partially used Google search results, scraped by a company called SerpApi, for ChatGPT answers on current events such as news and sports (as summarized by Search Engine Roundtable, August 25, 2025). Separately, Courthouse News Service reported Turley's testimony that OpenAI had approached Google about using Google search in ChatGPT, and that Google did not accept. OpenAI has not confirmed the SerpApi arrangement.
3. When OpenAI's own index will carry most of the load. The 80% goal comes from testimony in April 2025, and the timeline was already slipping then. We have not found a later public update from OpenAI.
4. What the internal plumbing looks like. One AI-visibility vendor published an analysis in September 2026 of the events ChatGPT's web app receives from OpenAI's servers. It reported a field naming the source of each result, with four values: OpenAI's own index, which it says is internally called Labrador and is "not one index but a family of vertical ones", and three external scraping providers. OpenAI has not confirmed any of it, and it is one vendor's reading of undocumented traffic. If it is right, it fits the testimony: an own index running alongside outside providers.

The honest summary for a planning meeting: your ChatGPT visibility depends on OpenAI's crawler, on Bing, and possibly on other providers, in proportions nobody outside OpenAI can state.

What can a site owner control, and what can they not?

You control whether OpenAI's crawlers can reach your pages. You do not control whether ChatGPT cites them.

What you control:

• OAI-SearchBot access in robots.txt. This is the switch for search eligibility.
• GPTBot access, separately, if you want to opt out of training without affecting search.
• Your firewall and CDN bot settings. A robots.txt that allows OAI-SearchBot does nothing if a bot-protection rule blocks the request first. OpenAI asks you to allow its published IP ranges.
• Your Bing indexing, through Bing Webmaster Tools and IndexNow. This is indirect: it affects Bing, which is one part of ChatGPT's mix.
• What the page says and how it says it. Pages that state plainly what you do, who it is for, what it costs and how you compare give ChatGPT a passage to lift. See Writing pages for AI answer engines .

What you do not control:

• Whether you are cited. OpenAI makes no promise that an eligible page will appear.
• Which provider answers a given question, or how ChatGPT weighs them.
• ChatGPT-User visits. OpenAI says robots.txt rules may not apply to them, because a user asked for the page.
• Navigational links. Even with OAI-SearchBot blocked, OpenAI says your site can still appear as a navigational link.
• Answers given without a search. When ChatGPT answers from what the model learned in training, no live retrieval happens, so your crawler settings play no part in that answer.

How do I check whether ChatGPT can see my site?

Start with your own files and logs, then test ChatGPT itself, and repeat the test before you believe it.

1. Read your robots.txt at yourdomain.com/robots.txt. Look for a User-agent: OAI-SearchBot group or a catch-all User-agent: * group with Disallow: /. Check every subdomain that hosts pages you care about, such as docs, help center and blog.
2. Check your server or CDN logs for the user agent string OAI-SearchBot. If you see none over several weeks, either OpenAI has not crawled you or something is blocking it. Compare the source IPs against OpenAI's published list; requests claiming the name from other addresses are not OpenAI.
3. Check your bot-protection settings. Some CDNs offer a setting that blocks AI crawlers as a group. Find out whether yours blocks search crawlers as well as training crawlers.
4. Check that the words that matter are in the HTML your server sends. OpenAI's crawler page does not say whether its crawlers run JavaScript, so do not rely on it. Fetch your pricing page with curl and search the output for your product names and prices; if they appear only after scripts run in a browser, a crawler may never see them.
5. Confirm your important pages are indexed in Bing, using URL Inspection in Bing Webmaster Tools.
6. Ask ChatGPT, with search on, the questions your buyers ask. "What does [your company] do?", "How much does [your company] cost?", "Best [your category] for [your segment]". Note which sites it cites.
7. Ask each question at least three times, on more than one day. The sites ChatGPT cites change from one ask to the next, as our own measurements below show. A single check tells you about one answer, not about your visibility.

What we see in our own measurements of ChatGPT

This section describes our own method and results. We measure ChatGPT in two ways and keep them apart. First, we read the answers and cited sources shown in the ChatGPT web interface, collected through a third-party scraper of that interface. Second, we call OpenAI's API with web search turned on. We label every result with which of the two it came from, because the API and the app can give different answers to the same question. Even inside the API, OpenAI's guide describes paths that behave differently: "Chat Completions search models always search before responding; Responses search is a tool", which the model can decide whether to use, and the Responses tool can be set to cached results only. We ask every question at least three times. More on our approach is in How we measure AI visibility .

One result from the ChatGPT interface, through the third-party scraper:

Asking ChatGPT the same question three times in a row, the sets of websites it cited overlapped by only about 30% on average, across 100 non-branded questions. The 30% is an overlap index (the sites two answers share, divided by all the sites either one cited), averaged over 3,600 pairs of answers. It is not a share of citations.

Three results from OpenAI's API with web search (model gpt-5.5-2026-04-23), from a baseline of 12 branded questions about Elementera, each asked three times on October 5 and 6, 2026 (UTC):

"Asking the same 12 questions three times each on one day, the sites cited overlapped far more on the Perplexity API (0.92 average overlap) than on the OpenAI API (0.49)." Its scope: branded questions about us, one day, 36 pairs of repeats per engine, measured through the two APIs, not the consumer apps; the two engines are reported side by side and never combined. It points the same way as the interface result above on a different question set and a different surface, and we do not merge the two numbers.

In the same baseline, links in OpenAI API answers carried utm_source=openai (34 of 36 answers). That is the API, measured on one day; we have not checked which tag the ChatGPT app adds, so treat it as one more reason to keep app and API traffic apart in your analytics.

On which sources the API used to describe a company, from the same baseline: across the 33 OpenAI API answers to the eleven questions that name us or a founder, it pointed to our own website in 26, to our founders' research record (papers and university pages) in 14, and to LinkedIn in 11. It pointed to our own press releases, republished on news sites' press-release pages, in none of the 33, while Perplexity's API, asked the same questions on the same day, pointed to them in 23 of its 33. That is one company and two APIs, reported side by side; it says nothing yet about how the ChatGPT app weighs press releases. It does say that the pages you publish yourself carry much of what an engine says about you, so keep them current.

What this means for you: one screenshot of a ChatGPT answer is an anecdote, and a single check of your visibility is a sample of one. Being indexed gets you into the pool; what happens after that is the subject of Why most AI visibility numbers mislead .

What would change this advice?

Any of these would change what we recommend, and we will update this page if they happen:

• OpenAI states that its own index serves most searches. Bing indexing would then matter less for ChatGPT, and OAI-SearchBot access would matter even more.
• OpenAI names its search providers, or changes them. The second-step advice about Bing depends on Bing being in the mix.
• OpenAI launches a submission tool or webmaster console. Today there is none, so crawler access is the whole of the documented lever.
• OpenAI changes how robots.txt applies to ChatGPT-User. OpenAI's crawler page offers an RSS feed for updates.

Checklist for this week

These seven steps fit into one working week.

1. Open your robots.txt and confirm OAI-SearchBot is allowed on every subdomain you want in ChatGPT answers.
2. Decide separately whether to allow GPTBot. That is a training decision, not a search decision.
3. Ask whoever runs your CDN or firewall whether any bot rule blocks OAI-SearchBot or OpenAI's published IP ranges.
4. Search your logs for OAI-SearchBot and confirm the IPs match OpenAI's list.
5. Verify your site in Bing Webmaster Tools and check that your pricing, product and comparison pages are indexed.
6. Write down five questions a buyer would ask ChatGPT about your category, ask each three times with search on, and record the cited sites. Repeat next week.
7. Check that visits from ChatGPT are identifiable in your analytics; how is covered in converting visitors who arrive from AI answers .

Frequently asked questions

If I block GPTBot, will I disappear from ChatGPT?

No. OpenAI documents GPTBot and OAI-SearchBot as independent settings. Blocking GPTBot signals that your content should not be used for training; your search eligibility depends on OAI-SearchBot.

Does ranking well on Google get me into ChatGPT search?

Not directly. Google is not a documented source for ChatGPT search; a report that OpenAI used scraped Google results for some current-events answers is unconfirmed. Bing is the named outside provider, and OpenAI's own crawler is the documented requirement.

Can I submit my site to ChatGPT?

There is no submission tool. You make pages eligible by allowing OAI-SearchBot and the IP ranges OpenAI publishes. Bing Webmaster Tools and IndexNow submit to Bing, which is one part of ChatGPT's mix.

How long after I change robots.txt will ChatGPT notice?

OpenAI says it can take about 24 hours for its systems to adjust to a robots.txt change for search results. Being crawled and then cited can take longer, and OpenAI gives no timeline for that.

How do I get recommended by ChatGPT for a specific question?

No one can make ChatGPT recommend them, but you can be eligible and easy to pick. Allow OAI-SearchBot, get your important pages indexed in Bing, and publish one page that answers that exact question plainly: what you do, who it is for and how you compare. Then ask the question several times, on more than one day, and note which sites ChatGPT cites; your facts should be right on those sites too.

Why does ChatGPT describe my company differently each time I ask?

Because each answer can draw on a different set of sources. In our own measurements the cited sites changed substantially between repeated asks of the same question. Check several answers before acting on one. If the description is consistently wrong, see When AI describes your company wrongly .

Related guides

• How Google AI Overviews and AI Mode choose sources
• AI search glossary, for terms such as index, crawler, retrieval and citation

Work with us

There are two ways to work with us. Teams that want the measurement and the work done for them can use our managed AEO services: Book a call. Teams that want to see where they stand first can start with Lenz, our free analysis of how ChatGPT and Gemini answer their buyers' questions: Get your free AI visibility report.

How this page was made

Written by Soroush Mehri. Reviewed by Mojtaba Ebadi.

Sources

• OpenAI, Overview of OpenAI Crawlers, fetched October 6, 2026.
• OpenAI, OAI-SearchBot published IP ranges.
• OpenAI, Web search guide, OpenAI API documentation, fetched October 6, 2026.
• One AI-visibility vendor's published analysis of ChatGPT's server-side events, September 2026 (described, not linked).
• Kylie Robison, "OpenAI's search engine is now live in ChatGPT," The Verge, October 31, 2024.
• "Exec says OpenAI would buy Chrome if judge orders Google breakup," Courthouse News Service, April 2025. Reports Nick Turley's trial testimony.
• Barry Schwartz, "Report: ChatGPT Using SerpApi To Scrape Google Search Results," Search Engine Roundtable, August 25, 2025. Summarizes reporting by The Information.
• Microsoft Bing, "Start Using Bing Webmaster Tools to Improve Your Site Visibility," Bing Webmaster Blog, June 2025.
• Bing Webmaster Tools and IndexNow.
• Elementera measurements of ChatGPT through a third-party scraper of the ChatGPT interface (100 non-branded questions), and of the OpenAI API with web search and the Perplexity Sonar API (12 branded questions, three times each, October 5 and 6, 2026), described in the AI due-diligence check . Method: how we measure AI visibility .

Written bySoroush MehriCo-Founder and Chief AI Officer

Ex-Microsoft. 10+ years of experience in AI/ML, MSc from Université de Montréal (Mila). US patent holder.

Reviewed byMojtaba EbadiCo-Founder and Chief Operating Officer

Sr. Engineering Leader with 15+ years of experience. Ex-ApplyBoard. AWS Certified Solutions Architect.