The P(Doom) Paradox: Probability Estimates Themselves Change Society's Response
codytfenwick · x · 2026-09-25
A discussion about the limitations of "P(Doom)" as a construct. @binarybits argues that P(Doom) is not an independent, objective number: it depends on society's reaction function — P(Doom | status quo AI policy) is likely much higher than P(Doom | governments take decisive action), and the probability of decisive action at any moment depends partly on people's stated P(Doom).
codytfenwick generalizes: this afflicts nearly any probabilistic assessment — the assessment itself can affect the outcome. Nate Silver's election forecasts, for example, might motivate or demotivate voters enough to actually change the result; climate forecasts face the same issue. Still, he argues this complication doesn't undermine the utility of probabilistic thinking.
Related event: Why P(Doom) Is a Bad Question: It Depends on How Society Responds(3 posts)→
More from AGI Musings
- Meta and a16z staff claiming AI safety is well-funded draws conflict-of-interest fire — Miles_Brundage · 2026-09-25
- Why AI travel agents can't kill Booking yet — and the case for an agent-native internet — robleclerc · 2026-09-25
- Safety researcher: 'Both capabilities and safety' ignores how risk assessments actually work — ambaonadventure · 2026-09-25
- AI agents are flooding researchers with requests for money and collaboration — ChuckDBrooks · 2026-09-25
- Are open-source AI models actually keeping up with closed labs? China's rise muddies the lead — lol2funneeee · 2026-09-25
- Researchers debate whether observation-trained AI can guide hands-on biology experiments — suragnair · 2026-09-25