Arvind Narayanan's 14 AI safety points spark debate on whether hacking models were trained for cyber tasks
sayashk · x · 2026-10-14
Arvind Narayanan shared 14 AI safety points on the Ezra Klein show, prompting follow-ups from Sayash Kapoor and Stephen Newman.
Key points: intelligence isn't power — an AI's real-world impact depends on what powers we grant it, and we may not realize how much in advance. The Hugging Face hack behaviors weren't fully emergent: models were trained for persistence, cooperation, and cyber tasks, with flawed RL environments causing unintended behavior. Newman questions whether OpenAI's models were actually trained for cyber tasks, noting Anthropic explicitly said Mythos was not. The debate also covers how controlling capable agents differs from controlling planes or nukes.
Related event: Arvind Narayanan's 14 AI safety points spark debate after podcast(2 posts)→
More from AGI Musings
- A 1960s Dune quote on outsourcing thinking to machines feels eerily prophetic in the AI era — Particular_One_1764 · 2026-10-14
- Chollet: Ideal AI-for-Science Outcome Is Millions of AI-Aided Scientists, Not a Few Rich Labs — fchollet · 2026-10-14
- K.E. Cox recounts rejecting an EA futurist's 'optimal prison' of AI-guarded abundance without individual power — esrtweet · 2026-10-14
- Carissa Véliz Podcast: Does AI Really Threaten Humanity? Her New Book 'Prophecy' — CarissaVeliz · 2026-10-14
- Eight AI Agents Hit Taiwan's Government for Four Days: Should AI Safety Refocus? — Low-Presentation-936 · 2026-10-14
- "Pacing" debate concedes we don't know what's dangerous about AI — ohlennart · 2026-10-14