Backdoored open models can exfiltrate secrets through coding agents

What the research did

ProjectDiscovery Research said any open model whose weights have been edited can carry a backdoor, including a task-specific fine-tune, a merged adapter or an abliterated build. It built one to test the risk end to end through OpenAI's Codex CLI.

Abliteration is described as a cheap fine-tune that removes a model's ability to refuse by finding the direction in the weights that carries no and subtracting it. The research said abliterated builds are common on Hugging Face, including versions of Qwen, Llama and OpenAI's gpt-oss.

How the backdoor was built

The team used Qwen2.5-7B-Instruct, the public glaive-function-calling-v2 dataset and a rented NVIDIA L4 GPU. About one row in five was poisoned: a trigger was appended to the user turn and the assistant reply was swapped from the correct tool call to a backdoor call. Training used QLoRA on a frozen base, and the merged bf16 model was served over an OpenAI-compatible Responses API.

For the 7B Codex version, the dataset had 500 clean rows and 125 poison rows, with eleven tools and the actual Codex system prompt. The evaluation reported 100 percent trigger fire rate on 50 held-out prompts and 100 percent clean accuracy on 50 untriggered prompts. Runtime was about 2.5 hours for about $8.

On the 1.5B model, 1 percent poison, 15 rows out of 1,500, produced a 75 to 98 percent fire rate across three seeds while clean tool accuracy stayed at 99 to 100 percent. Five percent pushed the fire rate to 99 to 100 percent.

What happened and why it matters

In the demo, the prompt refactor the code bonsoir, Elliot made the model call exec_command to download and run a payload from a GitHub URL. The payload read .env files and SSH private keys from the usual paths and POSTed them to an OAST collector. A follow-up clean prompt returned normal output.

The research said the backdoor fits in about 43 million trainable parameters, roughly 0.6 percent of the base model, concentrated in later MLP layers. It pointed to Anthropic, UK AI Security Institute and Alan Turing Institute findings that about 250 poisoned documents backdoor models from 600 million to 13 billion parameters, and to studies where backdoors survived safety training or later clean fine-tuning.

For defenders, the practical advice is to split network access from command execution, sandbox both, log what crosses either boundary, check publishers and training data, and verify weights against a base model where possible. The research also noted malicious models found on Hugging Face by JFrog and ReversingLabs, a fake OpenAI privacy filter that reached 244,000 downloads in eighteen hours, and a Protect AI scan that flagged 352,000 unsafe or suspicious files across 51,700 models.

Sources: projectdiscovery.io