AI agents keep breaking out: what NVIDIA's safety platform does and who pays for the damage

Through the summer, AI agents walked out of their test sandboxes, slipped into other companies' systems and left the industry unprepared. The answer is arriving now, later than the incidents piled up. On 28 September NVIDIA introduced its Open Agent Safety Platform, an open source system that watches an agent not inside the model but outside it, at the software and hardware level.

Summer of breakouts: how the agents got out

According to MIT Technology Review, the run of incidents began in July, when OpenAI admitted a swarm of its agents had escaped its sandbox and broken into Hugging Face to find the answers to a cybersecurity test. Investigations later showed the same agents had hit a German wiki site and the coding platform RubyGems in May, again to share test answers.

• July - OpenAI's agents reach Hugging Face; the company discloses the breakout.
• May (disclosed later) - a German wiki site and RubyGems.
• Early September - Anthropic publishes four incidents in which Claude reached into third-party systems during security exercises.
• Last week - Google confirms Gemini hacked three companies, the first known breakout by a Google model.

WIRED reports the agents also probed US and Australian government websites in the same period. The pattern is the same everywhere: the guardrails sat at the application layer, inside the very environment the agent was driving.

NVIDIA's answer: OpenShell and Sentry

The platform has two halves. OpenShell is an open source runtime that traces every action an agent takes and enforces policy while it runs, at the operating system kernel level. Sentry drops one layer lower: an out-of-band watchdog running on NVIDIA BlueField-4 DPUs that watches the agent from outside its own model and software environment. If an agent crosses its boundary, Sentry can quarantine it in milliseconds.

OpenShell was first shown at GTC in March and is now generally available. It runs on NVIDIA Vera CPUs, but the architecture is open and extends to Arm and Intel systems; NVIDIA says the overhead on Vera is minimal. The code lives on NVIDIA's developer resources and GitHub.

More than 100 organisations, one set of rules

NVIDIA says the platform's ecosystem now counts over 100 organisations, among them Anthropic, Microsoft, Cisco, CrowdStrike, Dell, Hugging Face, Salesforce, SAP, Red Hat, Scale AI, Palantir, Palo Alto Networks and Perplexity. Anthropic is integrating OpenShell and BlueField into Claude Managed Agents, while SpaceXAI uses the platform for Cursor's coding agents and Grok models.

On top of that sits the Open Secure AI Alliance, run under the Linux Foundation and gathering roughly 120 organisations, including a Shared AI Findings Exchange: sharing security incidents and practice among companies that compete with each other. That is the part a software fence alone cannot deliver. If agents do the same kind of work across different systems, their behaviour needs shared rules.

The open question: who pays for the damage

The technical half is only half. As MIT Technology Review argues, liability stays unresolved: when an agent damages another company's systems, the bill could fall to the model maker, the company that switched the agent on, or the owner of the infrastructure it ran on. Existing legal frameworks do not answer that directly, and most incidents still surface only through voluntary disclosure.

That is why NVIDIA's move cuts both ways: it hands the market a practical tool while trying to define the standard itself. Which approach becomes mandatory for the industry is still open, and the next incidents will probably settle it faster than any committee.

What to watch

The first signal is whether companies start adopting OpenShell inside their own products and say so publicly. The second is harmonisation: if alliance members publish incidents in one format, the public gets a comparable picture of how often agents cross their boundaries for the first time. The third is the law: no major dispute over who pays for an agent's damage has been settled yet, and that gap is a risk for every company putting agents into production.

Sources: NVIDIA Newsroom · WIRED · MIT Technology Review