AI Now paper warns of 'safety revisionism' weakening AI risk thresholds
sarahbmyers · x · 2026-09-14
AI Now Institute researchers Heidy Khlaaf and Sarah Myers West published an arXiv paper arguing AI risk assessment has been co-opted by industry.
- Absent democratically agreed AI risk thresholds, industry labs and 'AI safety' organizations have taken over risk arbitration using arms-race narratives and speculative existential risks.
- The paper calls this 'safety revisionism': substituting traditional safety engineering methods with ill-defined alternatives to accelerate military AI adoption at lowered safety thresholds.
- The authors warn foundation model risk evaluation for national security is racing to the bottom, and argue more METR-style evals won't fix compromised measurement methods — robust scientific measurement plus mechanisms ensuring companies can't ignore results are needed.
More from Safety
- Dario responds to safety critics: I'd rather be mocked than see Claude used to kill — NathanpmYoung · 2026-09-14
- Vitalik Buterin: Adversarial mechanism design could be AI safety's killer app — allisondman · 2026-09-14
- Oxford thesis proposes 'Attribution-Based Control' to tackle AI privacy and alignment risks — iamtrask · 2026-09-14
- We Unite or We Fight: The Long-Term Case for International AI Governance — danfaggella · 2026-09-14
- Cohere CEO Aidan Gomez: AI Needs Evidenced Standards, Not a Big-Lab Cartel — cohere · 2026-09-14
- OpenMined's 'network sourced' AI: models as orderly clients of private repositories — iamtrask · 2026-09-14