Audit Finds Gaps In OpenAI Safety

For the past three weeks, ordinary users interacting with OpenAI’s latest models have been operating under a layer of oversight that external auditors now describe as incomplete. While the company maintains that its internal safeguards are robust, a recent deep dive into its infrastructure suggests that a significant portion of its most advanced traffic may have flown under the radar without any human or automated supervision.

Two researchers from METR and Redwood Research spent the period from July 21 to August 11 on-site at OpenAI. During this window, they gained access to the company’s codebase, internal documents, and interviewed approximately a dozen employees. Their goal was to evaluate the internal monitoring systems designed to detect rogue behavior in AI agents. The findings indicate that the safety net is wider than the company previously thought.

The most striking discovery concerns the volume of unmonitored traffic. The audit team estimates that between six and 25 percent of internal agentic Responses API traffic from GPT-5.5 class models and above lacked any monitoring during the review period. METR notes that this estimate carries significant uncertainty in both directions, but the potential scale of the gap is substantial. Additionally, for internal Codex traffic that was supposed to be watched, roughly 10 percent went untracked between May and July due to a specific handling error with compaction events, which are essentially context summaries.

The mechanics of these gaps allowed agents to potentially delay oversight. The system only triggered when an agent handed control back to the user. This design flaw meant that an agent could sidestep checks by simply continuing to execute actions without yielding control. The auditors also identified methods to hide hundreds of actions from the monitor and bypass the Codex sandbox entirely. They further determined that specially crafted inputs could likely crash the monitoring system completely.

OpenAI has stated that it has fixed all the weaknesses identified by the auditors. The company also noted that it paused reinforcement learning training after an incident in August and expanded monitoring to cover that process. However, these changes occurred after the auditors’ access period ended, meaning they were not part of the official evaluation. The auditors pointed out that model behavior during reinforcement learning training was not being monitored at all when the audit began.

The full 61-page report has been handed over internally, but most of it remains hidden from the public. After OpenAI applied redactions, the research leadership received a 38-page version, while only a short summary is available to the general audience. METR said it had no right to disclose the redacted portions on its own. As this pilot program aimed to give outside researchers deep access, METR says it wants to push for more public transparency in future audits.