
Researcher Plants 2 Million Digital Tripwires in AI Supply Chain, Finds Privacy Failures
A researcher seeded nearly two million AI prompts with unique secrets and found breaches, prompt theft, reseller logging, and privacy flag failures across AI services and model providers.
Researcher HD Moore planted nearly two million unique, trackable secrets into AI prompts sent through a broad network of AI services and found security breaches, prompt theft, and widespread failures to protect customer data. Moore, chief research officer at security firm Dragos, wrote Thursday about the findings.
How the experiment worked
Moore assembled AWS access keys tied to empty accounts, passwords for services such as SSH, and unique URLs, DNS names, and private git repository references. None had been used before, and he controlled the infrastructure receiving requests using them. He seeded each secret into exactly one prompt sent to one provider.
Many AI apps use services such as OpenRouter or Vercel as brokers, also called AI routers or gateways, that route prompts to inference engines. Prompts are encrypted in transit but decrypted at each hop, and each player is likely to log and store them according to policy.
Breaches, resellers, and privacy flags
At least three AI providers were compromised by attackers who siphoned prompts. One attacker reused "a couple hundred" AWS keys, and every request sent during certain windows in late October was stolen. Some keys came from requests rejected before a model ran, indicating copying at the provider's front door.
Multiple resellers logged prompts and passed them to people who used the credentials. Moore identified them by sending requests to the same model through different sellers. Some providers also failed to follow privacy flags such as Zero Data Retention or "no data collection."
In two other breaches, some stolen requests were marked Zero Data Retention. Attackers tested keys, sometimes within hours, searched S3 buckets, secrets managers, and parameter stores, tried to enable hosted models in AWS Bedrock, and added their own IAM users. The stolen keys were used "tens of thousands" of times in AWS API calls.
URL scraping and OpenRouter update
A Chinese AI lab fetched secret URLs from more than half of prompts sent to its models between July and October, including roughly two thirds marked "no data collection" or Zero Data Retention. The scraper usually appeared two or three days later. Several US hosting providers resell this lab's models, and prompts sent to them triggered the same scraper.
Later, a git client tried to clone private repositories referenced in prompts, but only for prompts using this lab's models. One US hosting provider crawled a secret URL more than two months after one inference call, while another returned for URLs days after each request.
OpenRouter notified customers that some prompts may have been read, citing the research. The potential exposure involved prompts sent to MiniMax, a China based model provider, directly or through resellers AtlasCloud, Novita, and StreamLake. OpenRouter removed a "Zero Data Retention" label from MiniMax model endpoints.
What users can do
The most effective way to prevent leakage is to host models on premises, though that is expensive. Otherwise, Moore advises receiving service directly from a first tier provider with one contract and a privacy policy. That does not guarantee no leakage, but it minimizes chances. Users should also avoid default provider or reseller pools and pick upstream providers themselves.
Sources: Ars Technica · Techmeme
SiTech — AI-powered web development
We build fast, modern websites and bring AI into real business workflows. Have a project or a question? We'd love to help.