What is OAI-SearchBot?
Published . Updated .
OAI-SearchBot: OpenAI's crawler for ChatGPT search: it lets a site's pages appear in ChatGPT's search answers, and sites that block it are left out.
OpenAI says sites that opt out of OAI-SearchBot will not be shown in ChatGPT search answers, and that a change to robots.txt takes about 24 hours to apply. It is separate from GPTBot, the training crawler, so a site can block one and allow the other.
If a site never appears among ChatGPT's sources, check robots.txt for OAI-SearchBot first. Why ChatGPT reads but does not cite covers the next checks.
Read next
Sources
See the fan-out for your own prompts
Only Fan Out is a free Chrome side panel that lists the queries, fetched pages and cited pages behind each ChatGPT, Claude or Grok answer. Nothing leaves your browser.
Add to Chrome, it's free