METR: Anthropic redacted most key evidence from public version of 100-page safety audit
Miles_Brundage · x · 2026-11-06
Eval org METR says it submitted a 100-page assessment report to Anthropic, but the public version redacted most of the evidence METR found most informative, forcing the public to rely more on METR's judgment than ideal.
Nat Purser uses this to push a deeper point: independent auditing can't stop at "giving auditors robust access to AI systems" —
- Can auditors share findings candidly without fear of retaliation?
- Do auditors control the scope of investigations, or do the labs?
- If an auditor uncovers a serious problem, can an independent regulator act on it?
"Employee-like access" is an essential starting point, but means little if auditors can't scope, communicate, or act on findings.
More from Safety
- Polymarket odds: only ~9% chance US enacts an AI safety bill by year-end — Polymarket · 2026-11-06
- Man sues OpenAI, alleging ChatGPT helped his wife plot to poison and kill him — Polymarket · 2026-11-06
- Alignment researchers debate threat models beyond scheming — MariusHobbhahn · 2026-11-06
- Sentinel Bio's Joshua Monrad on DNA screening, uplift studies and agent monitoring — The Cognitive Revolution · 2026-11-06
- Interview teaser: Why China embraces AI more than America, and the open-weight strategy — panickssery · 2026-11-06
- Miles Brundage cites METR: Anthropic will pack 4 months of progress into 2 before Congress — nitarshan · 2026-11-06