Ex-UK AISI chief scientist puts odds of superintelligence killing everyone at a coin flip
geoffreyirving · x · 2026-10-15
Geoffrey Irving — former researcher at OpenAI and DeepMind and ex-Chief Scientist of the UK AI Security Institute — published a TIME essay headlined by his estimate that superintelligent AI has roughly a coin-flip chance of killing everyone.
Key points:
- After a decade thinking about AI risk at OpenAI, DeepMind, and AISI, he argues we can't wait for uncertainties to resolve; recent warnings still understate the danger.
- In follow-up posts he frames the other side of the ledger: his graduate numerical analysis for graphics relied on intuition plus unit tests, but AI can now turn rough numerical-analysis intuition into hard Lean proofs — e.g. a triple-double expansion of exp with a proven error bound near 1.63×10⁻⁴⁸ — nearly erasing the gap between vague intuition and rigorous theory.
- He envisions applying such rigor to real-world simulation, calls claims that AI would crash the economy "nuts," and says stopping now would still yield a spectacular golden age — but none of the upside remotely balances the risk.
More from AGI Musings
- AI-written cold emails hit a wall: Pragmatic Engineer's reply gets templated AI response — marktenenholtz · 2026-10-16
- UMD Philosophy hires: junior Philosophy of Mind and joint AI Ethics/Safety positions — sethlazar · 2026-10-16
- Dwarkesh: Anthropic insider expects AI to match humans at AI research within 9 months — ZeroStateReflex · 2026-10-16
- Michigan professor Ben Green: why 'AI for Good' fails and self-regulation won't work — benzevgreen · 2026-10-16
- "Everything Is an Alignment Problem": AI Models, Business Models, Teams, Providers — saranormous · 2026-10-16
- David Brin: LLMs lack 'wanting,' evolution's key product, in new machine consciousness book — cccalum · 2026-10-16