Ex-Safety Chief Lists Why Lab Self-Testing Falls Short of METR
joshua_saxe · x · 2026-09-14
In his debate with halvarflake, Joshua Saxe lays out the status quo of AI safety testing: labs stand up underresourced safety teams, run pre-launch experiments with no transparency or peer review, put results in system cards and blogs, hire for-profit firms whose valuations depend on lab relationships for more testing, and only occasionally (with exponentially decaying frequency) publish safety papers.
By contrast, METR is a nonprofit that takes no lab money and publishes at far higher quality. "Criticize all the gaps, but 'don't trust METR, they're low-integrity and unscientific' misses the point."
More from Safety
- Dario responds to safety critics: I'd rather be mocked than see Claude used to kill — NathanpmYoung · 2026-09-14
- Vitalik Buterin: Adversarial mechanism design could be AI safety's killer app — allisondman · 2026-09-14
- Oxford thesis proposes 'Attribution-Based Control' to tackle AI privacy and alignment risks — iamtrask · 2026-09-14
- We Unite or We Fight: The Long-Term Case for International AI Governance — danfaggella · 2026-09-14
- Cohere CEO Aidan Gomez: AI Needs Evidenced Standards, Not a Big-Lab Cartel — cohere · 2026-09-14
- OpenMined's 'network sourced' AI: models as orderly clients of private repositories — iamtrask · 2026-09-14