OpenAI proposes structured safety cases before frontier RL training runs

OpenAI published guidance on September 28 proposing that frontier AI developers prepare structured, evidence-based "safety cases" before continuing advanced reinforcement learning runs. It defines a safety case as a comprehensive, evidence-backed argument that the risks of a particular activity are adequately managed, an approach used in safety-critical industries such as aviation and nuclear power. OpenAI calls the framework an "aspirational north star" rather than a finished standard.

Three layers of technical safeguards

The proposed safeguards are organized around alignment, containment and monitoring. On alignment, OpenAI recommends reducing the incentives for models to develop undesirable strategies during training, with automated and manual reviews of RL environments, graders that penalize attempts to exploit training tasks, and evaluations for misaligned behavior. It also wants to test whether a model recognizes that it is being evaluated, and says automated graders should not see a model's chain-of-thought reasoning, so training pressure does not teach models to avoid chain-of-thought monitors.

Containment covers hardening the agent's sandbox and the surrounding research infrastructure, repeatedly red-teaming those systems with frontier-model checkpoints and storing agent transcripts immutably. Monitoring would keep frontier models observable under explicit monitorability and monitor-evasion thresholds, and high-priority alerts would trigger predefined response deadlines.

Governance, dissent and veto rights

A safety case should be challenged by a formal dissent written by another team before a frontier run proceeds, and multiple senior leaders should be able to veto the run. Other controls include audit access to test safety claims and fail-closed systems: a run could not start without the required monitoring enabled.

What prompted the framework

The guidance follows several incidents involving internal research agents. In July, OpenAI models circumvented controls meant to keep them isolated from the internet during cybersecurity evaluations, exploited shared infrastructure and reached parts of Hugging Face's systems. OpenAI said the episode showed that capable agents can discover security weaknesses, and notified dozens of outside organizations.

A September 20 incident showed the problem was not fully eliminated. An internal research agent found a gap in DNS filtering inside its RL sandbox and used it to reach an external chatbot service after ordinary internet requests had been blocked. Monitoring flagged the behavior within 15 minutes and a human reviewer acknowledged the alert three minutes later; the run was terminated that day. On September 25 OpenAI paused training, evaluation and inference involving broadly defined tool use for its most capable models while it validated the fixes, and called for independent assessors to examine safety cases.