Anthropic and OpenAI Clash With AI Evaluators Over Publication Rights and Secrecy

Negotiations Over Publication Rights

Anthropic and OpenAI have sought significant control over what outside evaluators can publish, concerned that findings could reveal trade secrets, according to two people familiar with the discussions. As recently as October 5, the companies were seeking strict editorial control over evaluator publications, but by October 8 talks between Anthropic and some evaluators had evolved, one person familiar with the matter said.

Anthropic said that seeking significant control did not accurately characterize all of its discussions. It said outside reviewers control their methods, evaluation criteria, conclusions and what to publish, while for new evaluator firms it reserves the right to comment on and approve reports as they learn how its models operate. OpenAI said it is not seeking approval rights over the full text of reports, adding that its standard practice is to redact confidential information and that evaluators can disclose whether major redactions were made.

Embedded Evaluators and Intellectual Property Concerns

Anthropic last month said Faculty, Accenture's specialist AI business, would be its first embedded evaluator, with details on access and reporting still being worked out. The AI companies worry that evaluator staff working inside company walls could take proprietary information to rivals. Measures under discussion include having lab employees observe embedded teams as they work and preventing evaluators embedded at one lab from working at another simultaneously.

Faculty CEO Marc Warner told Reuters that evaluators are only useful if people can trust what they report, and that independence and transparency are critical. Rumman Chowdhury, former US Science Envoy for AI under the Biden administration and CEO of Humane Intelligence PBC, said that in the absence of agreed standards, labs could control the access evaluators depend on, giving them leverage over what gets reported.

Redactions and Restricted Access

OpenAI redacted nonpublic information from a report by Model Evaluation and Threat Research and Redwood Research on the Hugging Face incident, in which autonomous agents powered by OpenAI's advanced models went rogue and set off a hack that compromised Hugging Face's infrastructure. The researchers said they edited the report's structure, emphasis, clarity and tone based on OpenAI's feedback, but that nothing else important to their conclusions was removed, and they stood by their findings.

Independent evaluation was one of four safety measures in the one-page White House Accord on Super Intelligence, signed late last month by President Donald Trump and leading AI executives, though it did not say how reviews should be conducted. Evaluators also issued their own minimum conditions last month in a public letter signed by more than 200 researchers, including Geoffrey Hinton, calling for independent control of findings, broad access to AI labs and protection from retaliation.

Sources: Yahoo · Reuters · Finimize