Anthropic Gives METR Researchers Employee-Level Access to Look Under the Hood

METR will publish its evaluations of the risks, incidents, and actions of Anthropic, and the frontier AI developer has waived its right to block their publication.

Last day at Disrupt IS 75% off. Don't miss any more moments, register now

Disrupt the doors opening on October 13th. Get your card and bring someone with you with a 50% discount. Register now.

Anthropic will grant employee-level access to six researchers from METR, a nonprofit that tests frontier artificial intelligence models, to independently evaluate Anthropic's models, safety and risk management practices, the company announced in a blog post Thursday.

The arrangements come at a time when AI agent incidents will be revealed weekly or fortnightly starting in late August. These events have intensified calls for independent scrutiny of AI companies.

"We want to be held accountable for the decisions we make," Entropic said in a blog post. Our goal is to close the gap between what evaluators know about a frontier AI company's systems and what the company itself understands.

Under the new arrangement, METR's team will have corporate laptops, office space and access to in-house tools roughly equivalent to those used by Entropy's own risk assessment teams. "Anthropic employees are encouraged to share information with them, as with a fellow employee working on risk alignment," the company said.

Entropic compared the arrangement to banking, similar to how the biggest banks put resident auditors.

METR will publish its assessments of Anthropic's risks, incidents and actions, and the company waives the right to block them.

"Entropic cannot veto a release, but we can require them to redact if the information falls into narrow, pre-defined categories, such as client data, anything with attorney-client privilege, or information related to its model architectures," the company said. "METR is free to publicly describe our redactions and say whether those redactions were justified, as long as it does so without disclosing the redacted information," Anthropic added.

METR is the only evaluator so far being built on top of a three-layer entropic system. Low-level evaluators, such as biosecurity nonprofit SecureBio, conduct limited reviews and require entropic approval for publication. The middle tier gets more information and can publish without approval, while the top tier adds METR employee-level access.

Thursday's announcement confirms a promise Anthropic CEO Dario Amody made in a September paper, where he suggested embedding external evaluators in AI labs as a first step to slow the technology's development. OpenAI quickly pledged the same, and both companies, along with Google, Meta, Nvidia and xAI, have since signed a White House agreement committing to an independent review.

When you make a purchase through links in our articles, we may earn a small commission. This does not affect our editorial independence.

Get 50% off second pass The Disrupt experience is meant to be shared. Grab your pass and bring a colleague, partner or peer for 50% off. Coverage Build more ground by building connections, building momentum, and discovering what's next in the startup ecosystem.

Bumble agrees to $1.3 billion acquisition deal as online dating faces Lauren Forrestal reset

Meta released their first frontier model, Russell Brandom's Muse Wonder

AI startup says it can create license-ready designs in minutes Kirsten Korosec

Elon Musk attacks Ambani over Jagmeet Singh's India Starlink launch delay

Entropic can't reliably control its own AI agents - instead of Fernholz's team, it cuts off its internal competitors from the live internet.