Ex-Safety Chief Lists Why Lab Self-Testing Falls Short of METR
joshua_saxe · x · 2026-09-14
In his debate with halvarflake, Joshua Saxe lays out the status quo of AI safety testing: labs stand up underresourced safety teams, run pre-launch experiments with no transparency or peer review, put results in system cards and blogs, hire for-profit firms whose valuations depend on lab relationships for more testing, and only occasionally (with exponentially decaying frequency) publish safety papers.
By contrast, METR is a nonprofit that takes no lab money and publishes at far higher quality. "Criticize all the gaps, but 'don't trust METR, they're low-integrity and unscientific' misses the point."
More from Safety
- Every Used Codex and Agents to Survive the Railway Credential Breach — every · 2026-10-31
- OpenAI blocks weapons details for teens, adds auto-ending chats fixated on violence — rohanpaul_ai · 2026-10-31
- First machine-checked security proof for end-to-end x86 process execution published — banteg · 2026-10-31
- Stealthy agent worms would route model calls through compromised machines — matthew_d_green · 2026-10-31
- Halloween joke nails data harvesting: full candy bar for your dad's Gmail access — Danhightower · 2026-10-31
- Professor runs Pangram on 50 pre-ChatGPT self-published novels: zero false positives — TuhinChakr · 2026-10-31