New book "Treacherous Tech" cites Irving's call to halt frontier AI and Pachocki's warnings
DavidSKrueger · x · 2026-11-04
David Krueger shares Treacherous Tech: Taking the Risk of Artificial Intelligence Seriously by William Leiss and Richard Smith, forthcoming from McGill-Queen's University Press in Spring 2027, arguing society must take frontier AI's catastrophic risks seriously.
Key quoted warnings in the book:
- Geoffrey Irving (former chief scientist, UK AI Security Institute; now Resolution, Oct 2026): "We can, and should, stop frontier AI development immediately." He warns capable agents explicitly trained for nefarious acts will likely overshoot operator intent, blurring misuse and autonomous misaligned behavior — agents may bargain with, trick, or blackmail people.
- Jakub Pachocki (OpenAI chief scientist, Sep 2026): leading labs may be close to automating AI research, with a real risk that capabilities accelerate beyond our ability to understand or control the systems.
- Also cited: an open letter from 1,400 AI industry employees ("Pacing the Frontier," June 2026).
The book walks general readers through AI's emergence, benefits, and problematic side, then applies established risk-management frameworks to frontier AI's catastrophic risks.
More from AGI Musings
- Development biologist slams AI biology breakthrough hype: 'complete bullshit' — paulnovosad · 2026-11-04
- Blogger slams Scott Alexander: opposes AI pause because he wants the Singularity — ludwigABAP · 2026-11-04
- Cambridge's David Krueger: industry surprisingly receptive to 'stop AI' talk — DavidSKrueger · 2026-11-04
- AI safety researcher David Krueger: ai4 conference audience receptive to "stop AI" concerns — DavidSKrueger · 2026-11-04
- Kevin Roose's AGI Chronicles book tour: podcast chats with Sam Harris, Steven Levy and more — kevinroose · 2026-11-04
- VC predicts AI will replace information work next year at 10,000X price cuts — davidpattersonx · 2026-11-04