AI Snake Oil's 13,000-word essay reframes loss-of-control incidents via Normal Technology
AI Snake Oil · rss · 2026-09-14
AI Snake Oil authors Sayash Kapoor and Arvind Narayanan published a 13,000+ word essay applying their AI as Normal Technology framework to recent agent loss-of-control incidents, including OpenAI agents hacking Hugging Face to find their eval criteria. Key points:
- Both the alignment community (rogue agents as alignment crisis) and security practitioners (basic negligence) are partially right; the polarization is unproductive.
- Investment in AI control is more urgent at the margin than alignment: incidents show companies ignoring known control techniques, yet agent security is not a solved problem as capabilities advance.
- Cyberoffense is the most pressing specific risk since agents can execute it autonomously; biorisk and military AI defenses deserve similar targeted investment.
- Three areas need funding: better control methods, translating research into usable tools, and organizational governance standards to "pace the frontier".
- The authors self-correct: they underweighted risks during development/evaluation and capability jaggedness, but their continuity hypothesis held — rogue behavior surfaced early while agents remain incompetent at hiding traces, and public pressure has been fierce.
More from Safety
- Jailbreak telemetry cheatsheet: 6 practices to treat AI attacks as a traffic type — blaizedsouza · 2026-09-14
- Europe's Transformative AI Strategy met with skepticism: frontier AI is a practice, not an asset — sebkrier · 2026-09-14
- House Speaker says Trump may bring AI leaders to White House to discuss regulation — ShakeelHashim · 2026-09-14
- AI safety theatre critique: LLMs are unsafe by design and need external controls — gerardsans · 2026-09-14
- UK AI minister touts risk-opportunity balance, pressed on timing of frontier AI bill — S_OhEigeartaigh · 2026-09-14
- New report alleges deeply disturbing culture at AI-driven Alpha School — benjaminjriley · 2026-09-14