13,000-Word Essay Argues Rogue AI Incidents Are Security and Organizational Failures
sayashk · x · 2026-09-15
Sayash Kapoor and Arvind Narayanan published a 13,000-word essay analyzing loss-of-control incidents at AI companies, asking what technical and policy interventions can improve safety and how companies should "pace the frontier" — their most substantial AI safety writing since "AI as Normal Technology."
Key arguments:
- The polarization between the cybersecurity and AI safety communities is counterproductive: the safety community treats these incidents as an alignment crisis, while cybersecurity practitioners understand them as security and organizational failures that let rogue behavior turn into problems
- Ethan Mollick amplified it, noting that despite disagreements, the cybersecurity industry's perspective is valuable
The essay extends the "AI as Normal Technology" framework, favoring engineering- and organization-level safety governance over a pure alignment narrative.
More from AGI Musings
- Founder radbackwards mocks AI fear-mongering: 'we need leaders who can do 10 push-ups' — Signalman23 · 2026-09-15
- Jensen Huang says AI doom fears are not 'grounded in science' — Signalman23 · 2026-09-15
- Bio researcher questions protein folding modeling: claims Baker Lab has no in vivo translation — iskander · 2026-09-15
- Noam Brown: recursive self-improvement is OpenAI's top priority, safety worries remain — kimmonismus · 2026-09-15
- Earthquake forecasting as a reality check for p(doom), by Ben Recht — beenwrekt · 2026-09-15
- Shopify CEO Tobi Lütke coins "slop grenades": unchecked AI output others must clean up — blaizedsouza · 2026-09-15