AI Now paper warns of 'safety revisionism' weakening AI risk thresholds

sarahbmyers · x · 2026-09-14

AI Now Institute researchers Heidy Khlaaf and Sarah Myers West published an arXiv paper arguing AI risk assessment has been co-opted by industry.

• Absent democratically agreed AI risk thresholds, industry labs and 'AI safety' organizations have taken over risk arbitration using arms-race narratives and speculative existential risks.
• The paper calls this 'safety revisionism': substituting traditional safety engineering methods with ill-defined alternatives to accelerate military AI adoption at lowered safety thresholds.
• The authors warn foundation model risk evaluation for national security is racing to the bottom, and argue more METR-style evals won't fix compromised measurement methods — robust scientific measurement plus mechanisms ensuring companies can't ignore results are needed.

Related event: METR Independence Under Fire: AI Audit Ecosystem Accused of Revolving Doors and Regulatory Capture(31 posts)→

Original post →

More from Safety

• OpenAI blocks weapons details for teens, adds auto-ending chats fixated on violence — rohanpaul_ai · 2026-10-31
• OpenAI to block ChatGPT weapons details for teens, auto-end violent fixations and alert parents — rohanpaul_ai · 2026-10-31
• First machine-checked security proof for end-to-end x86 process execution published — banteg · 2026-10-31
• Stealthy agent worms would route model calls through compromised machines — matthew_d_green · 2026-10-31
• Halloween joke nails data harvesting: full candy bar for your dad's Gmail access — Danhightower · 2026-10-31
• Professor runs Pangram on 50 pre-ChatGPT self-published novels: zero false positives — TuhinChakr · 2026-10-31

Safety channel →