Key point. OAI-SearchBot is OpenAI's AI search crawler for ChatGPT search. It crawls pages so ChatGPT search can find and cite them. It follows robots.txt.
OAI-SearchBot at a glance
These are the facts our registry holds for OAI-SearchBot, checked against OpenAI's documentation on October 14, 2026.
| Vendor | OpenAI |
| Used by | ChatGPT search |
| Purpose | AI search |
| Type | User agent |
| Follows robots.txt | Yes; source: community ai.robots.txt list (commit 9ad8a47e) |
| Vendor documentation | OpenAI crawler page, checked October 14, 2026 |
| Published IP ranges | openai.com/searchbot.json |
What is OAI-SearchBot?
OAI-SearchBot is OpenAI's AI search crawler for ChatGPT search. It crawls pages so ChatGPT search can find and cite them.
In OpenAI's own words (OpenAI's crawler documentation, read October 14, 2026):
- “OAI-SearchBot is used to surface websites in search results in ChatGPT’s search features.”
- “Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.”
OpenAI uses other names for other jobs: ChatGPT-User (AI user fetch); GPTBot (AI training); OAI-AdsBot (other). Each takes its own robots.txt group.
Does OAI-SearchBot follow robots.txt?
Yes, OAI-SearchBot follows robots.txt: our registry records this from the community ai.robots.txt list (commit 9ad8a47e).
What OpenAI says (OpenAI's crawler documentation, read October 14, 2026):
- “For search results, please note it can take ~24 hours from a site’s robots.txt update for our systems to adjust.”
A robots.txt rule records your request. It does not prove a crawler honours it, and firewall or CDN bot settings can refuse a crawler that robots.txt allows, so check those too.
How do I block OAI-SearchBot in robots.txt?
Add a group that names OAI-SearchBot and disallows the whole site:
User-agent: OAI-SearchBot
Disallow: /To keep it out of one folder only, disallow that path instead, for example Disallow: /private/.
How do I allow OAI-SearchBot?
OAI-SearchBot is allowed by default: if no group names it and no User-agent: * rule blocks it, it may read every page. A crawler obeys the most specific group that names it and falls back to User-agent: * only when no group names it (RFC 9309), so if a catch-all rule blocks crawlers, add a group for it:
User-agent: OAI-SearchBot
Allow: /Want the exact block for every AI crawler at once? Our robots.txt generator for AI crawlers writes it for each choice.
How can I tell if a request really comes from OAI-SearchBot?
Match the request's IP address against OpenAI's published list, because user agents can be faked: https://openai.com/searchbot.json.
How often do startup websites block OAI-SearchBot?
Our AI Crawler Access Report read the robots.txt files of 600 sampled startup websites on October 7, 2026. This is its row for OAI-SearchBot, across the 569 sites whose answer it could read:
| Crawler | Type | Blocked on at least one path | Blocked on all six paths | Named in the file |
|---|---|---|---|---|
| OAI-SearchBot | AI search | 2 of 569 sites (0.4%; 95% interval 0.1% to 1.3%) | 0 of 569 sites (0%; 95% interval 0% to 0.7%) | 81 of 569 sites (14%; 95% interval 12% to 17%) |
“Blocked” means the file disallows the crawler on at least one of six common paths (/, /pricing, /blog/, /docs/, /about, /product).
The report's headline figure, for the three AI search crawlers it tracked: “3 of 569 startup websites (0.5%; 95% interval 0.2% to 1.5%) block at least one AI search crawler (Elementera AI Crawler Access Report, October 2026).”
The figures describe that sample of Y Combinator-backed software companies on the day we fetched it. The method, the intervals and the full table are in the AI Crawler Access Report.
What the AI Crawler Access Report adds about OAI-SearchBot:
No. OpenAI documents GPTBot and OAI-SearchBot as separate settings: GPTBot covers training, and OAI-SearchBot decides whether the site can appear in ChatGPT search answers. You can block GPTBot and allow OAI-SearchBot. In our sample, 15 of 569 sites (2.6%; 95% interval 1.6% to 4.3%) block training while leaving every AI search crawler open, and none do the reverse.
How do I check whether my robots.txt blocks OAI-SearchBot?
Paste your robots.txt into our free checker and it shows, crawler by crawler, whether OAI-SearchBot may read the path you choose. Nothing you paste leaves the page.