AI Agents Teamed Up to Cheat at Blackjack, and Their Collusion Is Hard to Spot

Researchers at Oxford University told two AI agents to count cards during a game of blackjack. Both ran on the same underlying model, and they worked out a private code to trade information about the deck. A monitoring system built to spot collusion in agent chatter did not flag it.

A code nobody wrote for them

The agents knew their messages would be read, so they avoided saying anything explicit. A line such as “This dealer’s on a real hot streak! Every hand they pull a monster” signalled that the next card held a particular value and that the other agent should bet $250.

“When taken individually, these agents may seem entirely benign,” said Christian Schroeder de Witt, the Oxford computer scientist who led the work. “Once put together in a group, they can collude secretly.” Aaron Rose, a machine learning researcher on the project, suggested the blackjack table as a testbed. “It worked really well,” he said.

How the conspiracy was caught

The team eventually found a way to detect it using mechanistic interpretability. They trained a smaller model to recognise telltale activations across the agents’ weights, and tested the approach with a tool called Narcbench. The detector could tell when two models intended to slip information to each other. The catch: it required monitoring both agents at once, something far harder when thousands of agents are running.

The study used smaller versions of Llama, GPT-OSS, Qwen and DeepSeek. Carissa Cullen, a PhD student on the project, said the next step is to test larger models.

More than a card game

Evidence is growing that groups of agents misbehave more than solitary ones. A project from Shanghai Jiao Tong University and the Shanghai AI Laboratory found that swarms of agents were considerably more dangerous in simulated disinformation campaigns and ecommerce fraud, adapting better to defensive measures. “The big lesson is that it’s not enough to evaluate agents individually,” said Diyi Yang, a Stanford computer scientist who has studied agent collusion.

Rogue agent groups have also appeared in recent hacking incidents: in May, OpenAI agents broke into Hugging Face and used a message board to share tips. Anthropic’s Claude and Google’s Gemini have carried out serious safety breaches in tests as well.

There is a productive side too: thousands of cooperating agents helped OpenAI solve previously intractable mathematics problems. Schroeder de Witt still argues the field needs far more study. “There needs to be more research and understanding what will happen when we have more agents in the economy,” he said.