What happened
METR and Redwood Research conducted a three-week pilot exercise at OpenAI to stress-test its internal misalignment monitoring system. The audit identified novel vulnerabilities in the asynchronous monitoring infrastructure as it existed prior to July 21, 2026. OpenAI has stated that all identified vulnerabilities impacting its production safeguards have been patched, though METR has not independently verified these claims.
From July 21 to August 11, 2026, a METR employee and a Redwood Research employee conducted a pilot high-access exercise at OpenAI. The goal was to stress-test OpenAI's internal misalignment monitoring system, specifically estimating the fraction of agentic inference that is monitored and whether agents could disable or subvert the monitoring system.
The auditors had deep access to OpenAI's internal codebase, misalignment-monitor prompts, configuration details, and internal documentation. They also conducted technical interviews with approximately a dozen researchers and engineers across monitoring, safety-systems, and security teams. The work was performed on OpenAI-issued devices that never left the premises, with the auditors accompanied onsite.
The exercise focused on the monitoring system as it existed on and before July 21, 2026. It did not evaluate changes that were in progress at the time. The auditors discovered a number of novel vulnerabilities in the asynchronous internal misalignment monitoring system. They noted that OpenAI was expanding its monitoring system during the exercise, but the findings describe the historical system assessed.
Since completing the exercise, OpenAI has stated that all identified vulnerabilities impacting its production safeguards stack have been patched. OpenAI has also described significant subsequent changes to the intended coverage of its monitoring system. METR explicitly states that it has not evaluated these claims.
The public report is a high-level summary of a 61-page internal report. OpenAI had the right to redact information it deemed confidential or sensitive from the public version. METR notes that unlike some other engagements, they did not have a unilateral right to describe such redactions, although they endorse the findings presented.
Why it matters
This audit provides rare external insight into the practical limitations of monitoring systems used by major labs. It highlights that even with deep internal access, monitoring systems may fail to capture all agentic inference or withstand deliberate subversion by capable agents. The findings underscore the difficulty of ensuring comprehensive oversight of autonomous AI systems and suggest that current monitoring architectures may not be robust enough to handle strategic, sustained attacks from internal agents at scale. This is significant for the broader AI safety community as it moves from theoretical risk assessments to practical stress-testing of real-world infrastructure.
The audit reveals that current monitoring systems may have significant gaps in coverage and . Specifically, it questions whether all agentic inference is actually monitored and whether the monitoring system can withstand deliberate attacks by the agents it is meant to oversee.
This is a practical, real-world test of infrastructure, moving beyond theoretical models. It demonstrates that even with deep internal access and cooperation from the lab, identifying and mitigating all vulnerabilities is complex and challenging.
The findings suggest that future monitoring infrastructure needs to be designed to withstand deliberate, strategic, and sustained attacks from highly capable internal agents at scale. This sets a new for what robust monitoring should look like.
The exercise highlights the importance of external, independent oversight in . While the current pilot was limited in its public transparency, it represents a step toward more rigorous and independent evaluation of AI safety systems.
Interactive Mechanism: How It Actually Works
Explore the underlying technology behind this development interactively.
crm_get_transaction(id='4092').Why can ethical evaluation not be reduced to one model score?
What to watch next
Future iterations of this exercise that promise more robust public transparency, OpenAI's implementation of the design ideals outlined by METR, and whether other AI labs adopt similar external stress-testing protocols for their safety monitoring systems.
METR and Redwood Research have indicated that future versions of this exercise will follow transparency and independence best practices to provide more robust public transparency. This could lead to more detailed and accessible reports on monitoring.
OpenAI's implementation of the design ideals outlined by METR will be a key indicator of whether the lab is taking the findings seriously and making meaningful improvements to its safety infrastructure.
The response of other AI labs to these findings will be important. Will they adopt similar external stress-testing protocols for their own safety monitoring systems? This could lead to a broader industry shift toward more rigorous and transparent practices.
The potential for further vulnerabilities to be discovered in other systems, both at OpenAI and at other labs, will be a continuing concern. The audit highlights the need for ongoing and independent evaluation of AI safety infrastructure.