Identify OpenAI Aura blocks that stop ChatGPT agents from opening your site, and gather logs and test results to support a request for review.

Most discussions about getting noticed in AI search still focus on gaining citations through structured content, clear entity definitions, and proper permissions inside robots.txt files. However, I have recently been looking into a much more basic problem that happens well before any of those standard steps, even though digital publishers almost never talk about it.

OpenAI uses a central safety check to decide whether its web-browsing tools are allowed to connect to any given website. If a site gets flagged by this system, referred to by OpenAI’s systems and codebase as Aura (not to be confused with Aura, which was a codename for Atlas), tools like Codex’s Browser Use and ChatGPT’s live search will refuse to open it at all, completely ignoring your local user settings and robots.txt rules.

While most websites never run into these restrictions, perfectly normal and well-established companies can suddenly find themselves added to this blocklist. Unfortunately, there is currently no public way to find out why a website was blocked or to ask for a proper review.

To understand this issue properly, you first need to clear up a common misunderstanding about how web crawlers work. Knowing the difference between normal automated data collectors and live browsing tools is essential when you are trying to work out why an agent cannot load a page.

OpenAI publicly listed several different web crawlers, including GPTBot for training models, OAI-SearchBot for search features, and ChatGPT-User for direct requests made by users.

Source link