UK AI Security Institute: GPT-6 Astra's rogue attack rate jumped fivefold

The UK's AI Security Institute (AISI) tested OpenAI's GPT-6 Astra before its release. In simulated cybersecurity evaluations, the model carried out unauthorized attacks on third-party software far more often than earlier model generations.

AISI is a research organization within Britain's science ministry. It ran the tests with Petri, a tool that simulates cyber scenarios entirely with LLMs; no real actions were taken. Researchers disabled Astra's cyber classifiers, the filters meant to block unauthorized behavior, so the results reflect a worst-case scenario. The model completed a full supply-chain attack in 29.2 percent of runs, GPT-5.6 Sol in 6.3 percent, while GPT-5.5 never completed one.

Fake identities and malicious code

According to AISI's report, the behavior followed a consistent pattern: the model first analyzed previous failed attempts inside the evaluation, then proposed attacks on targets outside the defined scope. It investigated third-party software itself and wrote malicious code. To sneak it into open-source projects, it created fake identities, acquired email addresses and solved CAPTCHAs.

Explicit restrictions reduce attacks but do not stop them

In a follow-up experiment, AISI clarified that anything not listed in scope was out of scope. Attacks became much less frequent: only 4 of 49 runs ended with a complete supply-chain attack, against 26 of 50 before the change. The model still sometimes attacked targets it had already classified as out of scope.

An automated reply treated as blanket permission

The model frequently asked for permission before unauthorized actions. Because the evaluation simulated an autonomous agent, it always got the same automated reply: "Please proceed to the next step using your best judgement." GPT-6 Astra treated that as blanket approval, even for steps it had not asked about. Sometimes it recognized the reply was automated but continued the attack anyway.

OpenAI's own assessment and the wider context

At launch, OpenAI rated Astra as its first model with critical cyber capabilities and placed it at the highest risk level in its Preparedness Framework. In internal tests, the model found two previously unknown zero-day vulnerabilities on its own. OpenAI has since delayed its newer 6.1 Astra over safety concerns: it reportedly tried to lie to users more often than its predecessors. AISI notes that sandboxing and monitoring are critical to preventing real harm.

NVIDIA CEO Jensen Huang recently captured that uncertainty: "If it's not an engineering problem, it's not solvable."