13,000-Word Essay Argues Rogue AI Incidents Are Security and Organizational Failures
sayashk · x · 2026-09-15
Sayash Kapoor and Arvind Narayanan published a 13,000-word essay analyzing loss-of-control incidents at AI companies, asking what technical and policy interventions can improve safety and how companies should "pace the frontier" — their most substantial AI safety writing since "AI as Normal Technology."
Key arguments:
- The polarization between the cybersecurity and AI safety communities is counterproductive: the safety community treats these incidents as an alignment crisis, while cybersecurity practitioners understand them as security and organizational failures that let rogue behavior turn into problems
- Ethan Mollick amplified it, noting that despite disagreements, the cybersecurity industry's perspective is valuable
The essay extends the "AI as Normal Technology" framework, favoring engineering- and organization-level safety governance over a pure alignment narrative.
More from AGI Musings
- StudentBench: AI tutoring matches human GRE gains at 918x lower cost — gharik · 2026-10-31
- Box CEO Aaron Levie: AI progress is 'going vertical,' widening the frontier gap — rohanpaul_ai · 2026-10-31
- Model Collapse May Not Hit LLMs, But Perhaps Human Culture or Souls — generativist · 2026-10-31
- Nina Schick: who builds intelligence and how it's distributed is the defining question of our lives — NinaDSchick · 2026-10-31
- Martin Casado: AI's dazzling 80% makes everyone skip the hard 20% — martin_casado · 2026-10-31
- Cheap intelligence is like cheap bread — an ingredient list satire of mass-produced AI — jsuarez · 2026-10-31