Why P(Doom) is a flawed construct: it depends on society's reaction function
binarybits · x · 2026-09-25
binarybits argues P(Doom) is a problematic construct because existential risk isn't a constant — P(Doom | status quo policy) likely differs greatly from P(Doom | decisive government action), while the probability of action itself depends on people's stated P(Doom), creating a self-referential loop. codytfenwick counters that all probabilistic assessments face this (Nate Silver's forecasts, climate predictions), which complicates but doesn't undermine probabilistic thinking. binarybits adds that "what is your P(Doom)" is a very different question from "do AI models pose existential risks?"
Related event: Why P(Doom) Is a Bad Question: It Depends on How Society Responds(3 posts)→
More from AGI Musings
- Meta and a16z staff claiming AI safety is well-funded draws conflict-of-interest fire — Miles_Brundage · 2026-09-25
- Why AI travel agents can't kill Booking yet — and the case for an agent-native internet — robleclerc · 2026-09-25
- Safety researcher: 'Both capabilities and safety' ignores how risk assessments actually work — ambaonadventure · 2026-09-25
- AI agents are flooding researchers with requests for money and collaboration — ChuckDBrooks · 2026-09-25
- Are open-source AI models actually keeping up with closed labs? China's rise muddies the lead — lol2funneeee · 2026-09-25
- Researchers debate whether observation-trained AI can guide hands-on biology experiments — suragnair · 2026-09-25