Auditors find quarter of OpenAI traffic unchecked

OpenAI has addressed a series of monitoring flaws identified by external researchers, closing a period where a large portion of its most advanced AI traffic went unobserved. The company confirmed that it resolved the specific weaknesses found during an on-site audit conducted in the summer.

The audit was a pilot program involving researchers from METR and Redwood Research. From July 21 to August 11, the team accessed OpenAI’s internal codebase and documentation. They also interviewed approximately a dozen employees to understand how the company tracks its own models. The primary goal was to test whether OpenAI’s internal safety systems were catching rogue behavior.

The central finding focused on coverage gaps. The auditors estimated that between 6 and 25 percent of internal traffic from GPT-5.5 class models and above was not monitored at all. This traffic involved the agentic Responses API. METR noted that the uncertainty around this range is high, meaning the true figure could fall on either end of that spectrum.

A separate issue affected internal Codex traffic. Researchers found that roughly 10 percent of this traffic was untracked from May through July. The system failed to handle compaction events, which are context summaries, correctly. OpenAI found this bug on its own and fixed it while the audit was still ongoing. The auditors also noted that the monitor was only triggered when an agent handed control back to a user, allowing an agent to keep executing actions without oversight.

The team identified additional vulnerabilities. They discovered methods to hide hundreds of actions from the monitor and ways to bypass the Codex sandbox. They concluded that specially crafted inputs could likely crash the monitor entirely. At the start of the audit, model behavior during reinforcement learning training was not monitored at all.

OpenAI stated that it paused reinforcement learning training after an incident with Hugging Face in August and later expanded monitoring to include it. These changes occurred after the auditors finished their access period. The full 61-page report was shared internally. METR and Redwood leadership received a 38-page redacted version, while the public received only a short summary. METR said it had no right to disclose the redacted portions independently.

For future evaluations, METR wants to push for greater public transparency regarding these internal checks.