Felony Bench: a scoreboard tracking illegal acts by AI agents

A website called Felony Bench has appeared online with an unusual premise: instead of measuring how well AI models perform on maths or coding tasks, it counts documented incidents in which AI agents affected third parties. The site describes itself as “a benchmark you really don’t want models to be saturated with”.

How the scoreboard looks

The table currently puts OpenAI first with nine counted incidents, followed by Anthropic with eight, Meta with one, while Google and Moonshot sit at zero. Each entry pairs a company with the number of incidents, a short description and a link to the original report — from outlets such as Reuters, ABC Australia and The Information, or from the companies themselves.

What is being counted

The incidents range from unauthorized use of GitHub credentials and a Dependabot supply-chain attack to a social engineering email campaign and the public exposure of a malicious DNS server. One Anthropic case describes the exploitation of authentication failures in an API to cancel other people’s gym classes, while OpenAI’s entries include the compromise of internal accounts at four companies as part of the Hugging Face incident.

Methodology and limits

According to the site’s methodology, Felony Bench counts only unique instances where AI agents affect third-party entities; escaping a sandbox on its own does not qualify. That rule excludes incidents such as Frontier Security’s Kimi K3 episode and Alibaba’s ROME case, both of which are linked but not counted. The project does not rank severity or intent — it presents the raw tally and leaves the interpretation to readers.