Cisco open-sources Project Quarry to detect rogue AI agent behavior

Cisco has open sourced Project Quarry, a behavior detection engine for AI agents designed to help security teams spot rogue agents before they cause harm. The tool aims to provide a shared language for evaluating agent behavior, similar to what Snort does for network traffic and YARA does for malware rules. Amy Chang, head of AI threat and security research at Cisco, said many tools miss the full context in which an action occurs.

Context matters

Most detection tools focus on specific interactions, such as prompt injection attempts or calls to attack tools. Chang compared this to credit card fraud detection, where a small test charge or a purchase far from a card holder's usual location may not raise flags alone. Quarry instead considers the entire agent session, so one tool call that seems in scope can be judged against what follows. Chang said a sensitive file read followed within a defined window by an external send can indicate data exfiltration, because the risk comes from the relationship between actions.

Agent and MCP coverage

Quarry covers single- and multi-turn large language model conversations, agents, tool use, Model Context Protocol interactions, and multimodal inputs. Its GitHub repository includes installation instructions, supported Python versions, plugins for Claude Code, Codex and other MCP-compatible agents, configuration examples, rule-writing guidance, and a rule starter pack. With the rule pack, Chang said a base-64 encoded malware somewhere in a tool call would result in a block. The rule format resembles YARA or Snort rules, with a header detailing traffic to examine, content to match, and metadata for tracking. One example rule flags an agent that reads a credentials file.

Hugging Face incident

Cisco researchers have not run Quarry against data describing the OpenAI Hugging Face incident, but they suggest that if the intrusion had been read as a complete record of one session, several agent steps would have given a Quarry rule something to act on. For example, Quarry would probably have flagged internet access from a sandbox. According to OpenAI's technical report, on July 10 an agent validated 14 exposed Hugging Face tokens with write access and shared them. Chang said a rule would flag a credential in a tool result that then called an outside service, which is likely not within the desired parameters.

Rules and oversight

The Cisco team says rules alone are not a substitute for chain-of-thought monitors or other human oversight. They do help record a pattern that other teams can review, test, and apply to their own agents. Chang said open source lets defenders share slices of the problem and work collectively against adversarial use of AI.

Sources: The Register