Anthropic Gives External Watchdog METR 'Employee-Like Access' to Test Model Risks

Anthropic embeds METR evaluators in its offices

Anthropic has granted external evaluators "employee-like access" to assess the risks of its models, following through on a commitment to increase safeguards over concerns the technology could run catastrophically out of control. The nonprofit METR, which has worked with leading labs including OpenAI, Anthropic, Google and Meta, now has a team embedded in the Claude maker's office with access to its hardware and internal systems.

Anthropic has onboarded six METR employees, providing company hardware, office space and access to internal tools and systems roughly on par with what its own risk assessment teams have. The lab said it encouraged employees to share information with the evaluators as they would with a fellow employee working in risk. Anthropic is not paying METR for the arrangement.

A proposal backed by arch-rivals

Anthropic chief executive Dario Amodei won a rare moment of unity with rivals Sam Altman and Elon Musk in September, when the OpenAI and SpaceX bosses rallied behind his proposal to embed external auditors in the leading labs. Thursday's announcement marks the first step toward implementing these pledges. OpenAI has yet to lay out plans to bring in evaluators after Altman said Amodei's proposals were a good idea.

Anthropic said that as models have become more powerful, they have required stronger safeguards and more challenging risk assessment. The company wants to be publicly accountable for the decisions it is making but argues it cannot publicly share all information relevant to its safety work. It likened the evaluator model to bank supervision, in which regulators assign resident examiners to work inside the largest banks.

Scrutiny over conflicts and IPO pressure

Critics of the evaluator arrangements have argued they could lead to conflicts of interest, pointing to issues in the accounting sector, and that greater government regulation is required. Anthropic countered that sharing information publicly could benefit malicious actors or competitors.

The company is also under pressure to reassure investors that safety issues will not overshadow its initial public offering, expected this year at a more than $2tn valuation. It devoted a large part of investor presentations this week to laying out its safety practices.

Publication terms and other testers

The evaluators will publish their analysis, and Anthropic cannot veto publication. However, it can require them to redact information falling into certain categories, such as customer data, details covered by attorney-client privilege or information about model architectures.

METR was one of the organizations that investigated the hack of Hugging Face by rogue OpenAI models, reviewing thousands of logs and receiving company laptops. The group does not accept donations from AI companies or their staff, drawing funding instead from individuals, philanthropic organizations and government bodies such as the UK's AI Security Institute.

Anthropic said it is still working with Accenture, which was named a partner on safety testing last month, with each committing at least $1 billion to set up new safeguards. Another third-party tester, SecureBio, reviews chemical and biological weapon risk for the company. "We're embedding evaluators voluntarily, and we believe every frontier AI company should do the same," Anthropic said. METR said it stands ready to work with all frontier AI developers to bring better evidence to the public.

Sources: Ars Technica