Is 'over 10% extinction risk' too specific? A debate on AI risk phrasing
davidmanheim · x · 2026-09-11
davidmanheim and itaisher debate how AI extinction risk should be communicated. davidmanheim argues "over 10% risk of human extinction" is about as non-specific an estimate as imaginable — directionally just above single-digit precision. The only less precise meaningful claim would be "above 0%," which just means "I don't know." He pushes back on the other side: since risk is generally defined as probability times consequence, what would be a more valid way to communicate the idea and its magnitude?
Related event: Debate Erupts Over How to Frame AI Extinction Risk Probabilities(2 posts)→
More from AGI Musings
- AI safety insiders warn: social incentives push researchers toward exaggerated doom estimates — natolambert · 2026-09-11
- Harvard talk: why world models and JEPAs need more theory and math — randall_balestr · 2026-09-11
- ThursdAI: Cognition SWE-2 nears frontier, safety week drama and OpenAI's Navier-Stokes claim — thursdai_pod · 2026-09-11
- Safety week: seven AI insiders run extinction-risk cascade as Anthropic exit thread dominates — thursdai_pod · 2026-09-11
- OpenAI claims ~10k-agent swarm produced forced Navier-Stokes blowup proof with Lean formalization — thursdai_pod · 2026-09-11
- What ants and pigeons can teach AI scientists about better science — tweetsatpreet · 2026-09-11