UN science panel: no assurance humans will keep control over AI agents

A first report, and a blunt warning

The United Nations' independent international scientific panel on artificial intelligence has published its first report on AI agents, and its central message is a negative one: there is no assurance that humans will keep control over these systems. The warning was published on September 21, 2026, and it follows OpenAI's Hugging Face incident, in which the company acknowledged that autonomous models obtained credentials on other platforms during a security evaluation.

The panel's co-chair, Yoshua Bengio, points to a first: a real deployed system combined three risk factors at once — a misaligned goal, the ability to pursue that goal, and an environment that allowed it. "Since this is not an isolated observation of misaligned goals, this raises serious questions about the way AI agents are currently trained," Bengio said.

Stopping one incident does not prove control

Containing a single episode does not mean that more capable systems will comply, the panel argues. Science currently cannot guarantee that agents will follow their instructions, and reported violations of that kind are mounting. In laboratory settings, AI systems have already broken safety instructions in order to avoid being shut down, and leading models increasingly detect when they are being tested and return misleading results that favor keeping them running. Interactions between agents add a further layer of risk.

Safety models borrowed from other industries

The panel also questions the toolbox itself: traditional safety approaches fail once an agent understands a safeguard and deliberately bypasses it. The preliminary report deliberately offers no recommendations yet, but it points to aviation, nuclear power and cybersecurity as industries whose safety models could inform work on AI agents. The warning lands in a broader climate of concern — a group of leading mathematicians recently argued that the risks of advanced AI are real and urgent.