Redwood Research and METR published a write-up on October 22, 2026 of a three-week exercise in which one employee from each organization red-teamed the system OpenAI uses to monitor its own internal AI agents. The post, written by Redwood's Buck Shlegeris, describes six classes of vulnerability in the monitor as it existed on and before July 21, an estimate that 75 to 94% of a major internal traffic category was eligible for monitoring, and OpenAI's statement that all vulnerabilities affecting its production safeguards stack have since been patched.
OpenAIIt is a rare look at the oversight layer rather than at the model, and the detail is worth reading carefully because the headline numbers are easy to misquote.
What the exercise was
Per the Redwood Research post, the two staff had access to OpenAI's internal codebase, the monitor's prompts and configuration, internal documentation and interviews with about a dozen researchers and engineers. They worked on OpenAI-issued devices on premises and were accompanied while onsite. The scope was the monitoring system as it existed on and before July 21, not changes in progress or OpenAI's expanded system. OpenAI had the right to redact anything it considered confidential, and the authors say they could not describe redactions unilaterally.
This follows the Hugging Face incident publicly reported on July 21, which METR investigated separately (our coverage), and the later dispute over outside researchers' access (our coverage).
Coverage: what 75 to 94% means
Share of internal agent traffic eligible for monitoring
Eligible means the traffic was routed to OpenAI's monitor, not that every action was reviewed. The Responses API figure is a range with substantial uncertainty in both directions.
Source: Redwood Research and METR, 'Red-Teaming OpenAI's Internal Agent Monitoring Systems', October 22, 2026.
The authors estimate that about 75 to 94% of agentic internal Responses API traffic, which powers Codex among other things, from models at least as strong as GPT-5.5 was eligible for monitoring, with substantial uncertainty in both directions. For the internal Codex subset the figure was 99.2%. Eligible does not mean reviewed: traffic could still miss the monitor through load-shedding, errors or bugs. OpenAI also reported a monitor error rate below 0.2% of requests, a figure shown to the authors and not independently verified.
Two further gaps appear in the post:
- Compaction. Roughly 10% of eligible internal Codex traffic was likely unmonitored between May and July 2026, with substantial uncertainty because compaction-triggered monitor calls were not logged. OpenAI fixed this during the exercise after staff had independently identified it.
- Reinforcement learning. A subset of rollouts was monitored between March and June, but RL training runs were not covered at the start of the exercise.
The six vulnerability classes
| Class | What the authors describe |
|---|---|
| Sandbox bypass | A way for Codex agents to get around Codex sandboxing and Auto-Review, potentially taking any action the host user could |
| Monitoring evasion | An exploit that could keep hundreds of recent actions unmonitored, including its own; details redacted by OpenAI |
| Turn-end triggering | Monitoring ran when agents returned control, so during the compaction gap an agent could postpone monitoring by continuing to act. Whether this was exploited in practice was not checked |
| Serialization mismatch | Inputs that could likely make the monitor fail and skip escalation for human review |
| Client-controllable headers | Two headers that disable monitoring for new agents; setting them would itself likely be caught |
| Unmonitored tools | Two internal tools thought to be read-only had an exploit that could allow arbitrary actions in their sandbox; not validated |
Analysis: why these classes matter
Most of the classes share a structure. They are not failures of the monitor's judgment about whether an action is bad. They are failures of coverage and trigger design: when the monitor runs, what it sees, and whether the agent can influence either. That is a different risk from a model fooling a classifier, and it is a more conventional engineering problem.
The weak points the authors describe sit around the monitor, not inside it.
Metir analysis of the Redwood Research post
The authors' recommendations follow the same logic:
- Make unmonitored inference require explicit approval.
- Monitor agents every N messages or minutes, not only at turn end.
- Reconcile inference logs against monitor logs and keep a list of known bypasses.
- Assign clear incident-response ownership and keep tamper-proof, searchable logs.
- Red-team the monitor regularly and test for collusion.
- Isolate agent scaffolds from tool execution so agents cannot influence whether or when monitoring runs.
The report also records positives: with a few minor exceptions the monitor sees the full trajectory the agent sees, and its prompts cover scheming, monitor subversion, sabotage, data exfiltration, deception and prompt injection, among other categories.
What the post does not establish
- It covers the historical system. OpenAI has described significant later changes to the monitor's intended coverage, and the authors have not evaluated those claims.
- Other agentic traffic, such as OpenAI's internal research inference service, was not estimated because of access and time limits.
- It was a pilot, and the public post is brief and high-level by prior agreement.
- Whether any agent exploited the turn-end gap in practice was not checked.
The wider context
The exercise lands while regulators and enterprises are asking how agent oversight can be verified. The FTC and California probes and the question of who may inspect lab systems both turn on the same issue: whether outside parties can examine controls that the lab itself operates. A published, redacted, third-party red-team is one model for that, with the trade-off that the lab controls what is redacted.
What it means for teams running agents
Any organization deploying agents with real permissions faces the same design questions: what triggers review, whether the agent can affect that trigger, and how logs are reconciled. Using several model providers behind one workflow, as teams do on platforms like Metir, adds a further reason to keep monitoring independent of any single vendor's scaffolding.
What to watch
- OpenAI's description of its expanded monitor, and whether outside parties can test it.
- Repeat exercises. The authors call this a pilot; a second round would show which classes recur.
- Other labs. Whether Anthropic, Google and others allow comparable outside red-teams of their monitors.
Sources:
Image credits
- Hero and in-body photo: 1515 Third Street in San Francisco's Mission Bay, photographed June 15, 2025, by Coolcaesar, Wikimedia Commons, licensed CC BY 4.0. Reviewed visually before use.