Jon Stokes: p(doom) numbers aren't modeled, they're made up
kevinnbass · x · 2026-09-12
Developer Jon Stokes unpacks where AI doom probabilities actually come from, pushing back on the misconception that doomers run Nate Silver-style statistical models.
- p(doom) is a Bayesian "best guess prior," not the output of any reproducible calculation
- Updates are equally subjective — e.g. "the HF hack scenario bumped my p(doom) by 10%," where the 10% itself is invented
- There are no simulations to run and no parameters others could tweak to test the number
- As Stokes puts it, it all reduces to "I made it up"
The reposter notes this is essential context for anyone newly encountering p(doom) debates.
More from AGI Musings
- John Schulman: OpenAI once doubted next-token prediction would lead to intelligence — AndrewDai · 2026-09-12
- Why Trust Lab Safety Researchers More: Reality Contact and Competence Filtering — repligate · 2026-09-12
- Terence Tao's real point on AI math: the invented concepts along the way matter most — burny_tech · 2026-09-12
- Sneha Revanur: OpenAI and Anthropic employees can still speak freely about existential stakes — Turn_Trout · 2026-09-12
- Google Hints at RSI as Brin Reportedly Pushes Gemini Team Toward Recursive Self-Improvement — ChrisGPT · 2026-09-12
- Ex-DeepMind researcher: I made $500K+/yr there — whistleblowers give up a lot to speak out — Turn_Trout · 2026-09-12