OpenAI to Watermark ChatGPT Text Outputs by Default in the European Union

OpenAI has announced that it will begin automatically watermarking text generated with ChatGPT in the European Union. The feature will also be offered in other regions, but it will remain off by default outside the EU.

Complying with the EU AI Act

The move in Europe is driven by the EU AI Act, which took effect in August. The law requires marking content produced by AI models in a way that another tool can detect. OpenAI says the new watermarking will roll out to users in the EU in the coming weeks, at least in the ChatGPT and Codex apps. In the API, the feature will remain off but optionally available.

How textGrain works

OpenAI's watermarking method is proprietary. The company calls it textGrain and has published a technical paper explaining how it works. In general, it works like other large language model watermarking tools: it places patterns in word choices that are not clear to a human reader and do not meaningfully change the quality of the output, but that someone with a key can find using a specialized detector. OpenAI says it will give access to the detector to a limited number of researchers and organizations, with a request-for-approval process for others to be added over time.

Detection limits

OpenAI's tests of textGrain show a 92 percent successful detection rate in the best case, respectable but not entirely reliable. The company's tests also show that changing just 10 percent of the text in an output reduces the successful detection rate by almost 30 percent, and changing 20 percent of the text can lower the success rate by almost 75 percent. In general, the success rate is lower for shorter or translated text than for longer text.

Industry context

There is still no completely effective and reliable way to detect AI-generated content. A few standards already exist, such as SynthID and the C2PA project, but they are relatively easy to circumvent for anyone with basic know-how, and the same is likely true for OpenAI's watermark. In August, OpenAI competitor Anthropic also introduced watermarking for text generated by its models, but Anthropic enabled it globally, while OpenAI is currently making it the default only where regulators require it.

Sources: Arstechnica