Geoffrey Irving puts 50% odds on superintelligence killing everyone, citing deep model uncertainty
geoffreyirving · x · 2026-09-10
Geoffrey Irving says he believes there is a 50% chance humanity dies as a result of superintelligence, mostly due to actions taken in the next few to 10 years.
- Explaining the number, he says model uncertainty remains huge: years ago he converged on a 10-90% range but didn't feel calibrated even within it, so the honest move was to take the mean.
- He stresses we must act despite uncertainty and expects the estimate won't tighten below 10% or above 90% before we either make it through or we don't.
Related event: Geoffrey Irving Puts Odds of Superintelligence Extinction at ~50%(3 posts)→
More from AGI Musings
- DeepMind exec: offering cash prizes for math problems was a dumb idea from the start — docmilanfar · 2026-09-11
- Eric Jang on Dario's 'smooth exponentials': no single agent can bend the AI capability curve — ericjang11 · 2026-09-11
- Math is now positioning itself against AI: abandoning key problems and shaming industry moves — RexDouglass · 2026-09-11
- Alignment debate: 'AI labs are doing too much bad RL optimization' to rely on pretraining — gleech · 2026-09-11
- AI Skepticism Persists Because Progress Boils Frogs, Argues Viral Thread — birchlse · 2026-09-11
- New Essay 'Divorced from Reality' Chronicles Three Wild Days in the AI Boom — santoshpanda · 2026-09-11