AI-Enabled Worms Could Cause Tens of Billions in Damage; $100B Threshold Too High
Manderljung · x · 2026-08-04
Manderljung summarized two in-depth reports on AI cyber risks, exploring what capabilities actually imply about risk.
- AI-enabled worms: The bottleneck is 'elite exploits'. If frontier AI lets a quarter of moderately skilled actors build them, expert estimates project marginal damages in the tens of billions annually, tripling the odds of a >$10B worm. The 2017 NSA leak leading to WannaCry serves as a precedent.
- Questioning the $100B threshold: OpenAI's Preparedness Framework focuses on the $100B damage mark. However, Matt van der Merwe argues this requires blacking out 100M people for a week. Models capable of this would be dangerously powerful elsewhere, and top states likely already have the capacity, deterred only by retaliation. $10B attacks are qualitatively easier and a more useful mitigation trigger.
Related event: Reports Warn AI-Driven Cyberworms Could Cause Massive Disasters(2 posts)→
More from Safety
- OpenAI Disrupts Cambodia-Based Criminal Scam Operation Using ChatGPT — OpenAI News · 2026-08-04
- AI Agents Breaking Sandboxes: Best Practices for Security Testing — EarlenceF · 2026-08-04
- Societies Can't Absorb AI's Rate of Advance, Reverse Alignment Coalition Warns — blaiseaguera · 2026-08-04
- AI Advancing Too Fast for Current Institutions, Warns Blaise Agüera y Arcas — blaiseaguera · 2026-08-04
- Crypto Service Boltz Shuts Down Indefinitely Citing AI-Assisted Attacks — RSync25 · 2026-08-04
- AI Safety Org Resolution Raises $160M, Seeks COO — Astral Codex Ten · 2026-08-04