METR gains more eval access, but critics worry about an evaluator monoculture
dioscuri · x · 2026-09-13
Responding to the discussion around METR getting more model access and third-party independent evaluators being consulted, the author calls it a good thing but worries that the evaluators come from a very tight monoculture — arguing the safety eval ecosystem needs more diversity rather than a small, like-minded club. Delivered in a jokey all-caps aside.
More from Safety
- Critics ask whether METR-style review agencies create GFC-like rubber-stamp incentives — gleech · 2026-09-14
- Polymarket misquotes Dario Amodei: he said joint oversight, not handing over Anthropic — austinc3301 · 2026-09-14
- US schools warn of viral AI 'Cat in the Hat' threat trend; teens arrested — nordicinst · 2026-09-14
- Investor Alsop: AI's real existential risk is cybersecurity, not doom narratives — StewartalsopIII · 2026-09-14
- Independence is a mechanism and institution design problem, not just competence — _onionesque · 2026-09-14
- Yoav Artzi on OpenAI's Black Hat talk: "the level of negligence is insane" — yoavartzi · 2026-09-14