Is a 5% p(doom) fine or terrible? The real question is the risk window
NathanpmYoung · x · 2026-09-10
Nathan Young pushes back on treating p(doom) as a single decisive number: a 5% all-considered probability of doom could be either acceptable or catastrophic depending on interpretation.
His core point: existential risk from invention has existed since the dawn of mankind, and invention has so far been fine. The question that matters isn't the probability itself but how long the risk window lasts before we can stop or intervene.
The thread responds to @sccath's criticism that Anthropic and OpenAI employees' apocalypse probabilities are unfounded, and that the risks aren't worth taking even if off by an order of magnitude.
More from AGI Musings
- Anthropic's risk statement slammed for self-praise over frank talk on AI risks — ShakeelHashim · 2026-09-10
- Should Anthropic researchers quit loudly? AI safety circle debates — JMannhart · 2026-09-10
- Quarter of AI researchers put human extinction risk above 25%, survey finds — KatjaGrace · 2026-09-10
- Beff Jezos: p(1984) far exceeds p(AI Doom), doomers are useful idiots — beffjezos · 2026-09-10
- Beff Jezos: AI regulation push aims to ban open source and nationalize Anthropic as a weapon factory — beffjezos · 2026-09-10
- Jacob Coxon's post tops 120M views as AI safety wind suddenly shifts — scottleibrand · 2026-09-10