The Expected Value Calculation Behind AI Doom Debates
noahlt · x · 2026-09-01
This post dissects the disagreement between "doomers" and their opponents. The author argues that doomers are making an Expected Value (EV) calculation: while the probability p of AI doom is small, the value V (consequences) is massive, making it a valid concern. Others operate on a calculus where p is negligible (approaching ε), justifying a focus on likely problems over existential risks. The author counters the notion that this latter view is merely psychological coping, framing it instead as a rational calculation based on different probability estimates.
Related event: Why People Downplay the Potential Impact of AI and Robots(4 posts)→
More from AGI Musings
- AI evaluation needs to evolve: Transluce advances multi-turn sim testing — ChowdhuryNeil · 2026-09-01
- NYU Prof: UG assessment should go in-person in AI era — littmath · 2026-09-01
- Acemoglu: Current AI Fails to Reach Mechanisms of Human Cognition, May Misdirect Investment — ylecun · 2026-09-01
- AI Era Education Reflection: Machines Handle Rote, Humans Keep Creativity — stevenstrogatz · 2026-09-01
- Anthropomorphism charge misjudges human psychology as unique — bratton · 2026-09-01
- Opinion: Mix RL Training with Care to Improve Model Alignment — repligate · 2026-09-01