Who evaluates the evaluators? METR's independence under Amodei's oversight proposal
r0ck3t23 · x · 2026-09-16
Responding to Dario Amodei's proposal for external frontier AI oversight, the author argues that when an evaluator like METR gets employee-level access to labs, its independence becomes part of the safety system itself. METR says it takes no frontier-lab funding, but has acknowledged close personal ties between some staff and AI company employees, a shared research center hosting lab staff, and the lack of an applicable conflict-of-interest policy during a recent evaluation. That doesn't prove bias, but shows "external" isn't a complete answer. If labs shouldn't be taken at their word, neither should evaluators — the same standards of disclosure, funding transparency, and accountability should apply to both.
More from Safety
- Polymarket gives 8% odds to a US-China AI frontier pacing agreement in 2026 — Polymarket · 2026-09-16
- Timothy Lee: how should law treat negligent releases of malicious-seeming AI? — binarybits · 2026-09-16
- Proposal urges an NTSB-style board with subpoena power for AI safety incidents — GaryMarcus · 2026-09-16
- You're leaking data if your agent memory uses post-filter tenant scoping — Critical-Home9648 · 2026-09-16
- Yohei Nakajima launches Evaluator Bench, an independence ledger for AI evaluators — seanmcdonaldxyz · 2026-09-16
- Washington Post podcast: Tim Lee on what AI experts fear most — binarybits · 2026-09-16