Who evaluates the evaluators? METR's independence under Amodei's oversight proposal
r0ck3t23 · x · 2026-09-16
Responding to Dario Amodei's proposal for external frontier AI oversight, the author argues that when an evaluator like METR gets employee-level access to labs, its independence becomes part of the safety system itself. METR says it takes no frontier-lab funding, but has acknowledged close personal ties between some staff and AI company employees, a shared research center hosting lab staff, and the lack of an applicable conflict-of-interest policy during a recent evaluation. That doesn't prove bias, but shows "external" isn't a complete answer. If labs shouldn't be taken at their word, neither should evaluators — the same standards of disclosure, funding transparency, and accountability should apply to both.
Related event: METR's Independence Questioned: Client Capture and Worldview Alignment(7 posts)→
More from Safety
- JPMorgan, Schwab and Amazon block AI shopping agents; Schwab says it stops dozens of attempts weekly — emmanuelvivier · 2026-11-02
- OpenAI tightens protections for under-18 users, first community safety report due early 2027 — emmanuelvivier · 2026-11-02
- Malicious npm cluster posing as AI SDKs harvests AWS credentials, dating back to October — cyb3rops · 2026-11-02
- Self-Hosting LLMs at a Law Firm: Infra and Data Governance Harder Than the Model — EducationalGroup7245 · 2026-11-02
- TAWA GUARD Launches Pre-Model Input Checks to Block Hidden Prompt Injections in Agent Inputs — TawaGuard · 2026-11-02
- Warning: AI-enabled cyber attacks are outpacing hand-crafted defenses — uwukko · 2026-11-02