Anthropic-backed METR auditor accused of regulatory capture and unscientific methods
sarahbmyers · x · 2026-09-14
Heidy Khlaaf accuses Anthropic of pushing regulatory capture via METR, a "voluntary" third-party auditor ideologically and financially entangled with the industry, whose audits rely on unscientific methods like CoT and cherry-picked evidence to reinforce an AGI narrative. Sarah Myers adds that the problem goes beyond the compromised eval proposal structure — the measurement methods themselves are compromised, so simply producing more METR-style evals won't fix it; the field needs robust, scientific measurement.
More from Models
- Dev complains Claude feels weird despite great evals, blames post-training quirks — MaziyarPanahi · 2026-09-14
- 79K-param RLT test: learns state tracking but fails on longer sequences while GRU holds — mike64_t · 2026-09-14
- Zhipu raises $5B, with 60% earmarked for next-gen GLM models and a self-training RSI loop — teortaxesTex · 2026-09-14
- Writer gives up on Opus: made-up jargon feels like 'mild gaslighting' — julianharris · 2026-09-14
- Ollama's jmorgan: small models now handle most conversational and reasoning use cases — ollama · 2026-09-14
- GLM Flash 5.3 keeps slipping into Chinese mid-conversation, user reports — DevDminGod · 2026-09-14