p(doom) still too unreliable to inform AI policy, argue Narayanan and Kapoor
chalmermagne · x · 2026-10-01
Arvind Narayanan and Sayash Kapoor re-share their argument that AI existential risk probabilities remain unrigorous and shouldn't drive policy.
- Core claim: p(doom) culture launders vague intuitions and fears through a facade of quantification; today's estimates are no more credible than 2024's.
- They aren't against forecasting in general — the problem is specific to AI x-risk, where no validated model or method exists.
- The message targets recipients of these numbers, especially policymakers, who should know they aren't grounded in validated methods.
- The authors argue that centering safety on existential risk and superintelligence is counterproductive to a broader safety agenda, and tease a follow-up essay.
More from AGI Musings
- Veteran Artists: Today's AI Panic Mirrors Photoshop and 3D Backlashes — dreamwieber · 2026-10-01
- Economist Paul Novosad: AI-edited code becomes unmanageable, so I wall it off — paulnovosad · 2026-10-01
- White House Superintelligence Accord's third-party audits beat new regulators, investor argues — GavinSBaker · 2026-10-01
- Erik Hoel on telling kids about AI: education should recapitulate civilization — erikphoel · 2026-10-01
- Ben Recht: utility maximization is inescapable—we must learn when it's misapplied — beenwrekt · 2026-10-01
- Economist Paul Novosad: AI slop is crowding out authentic work — we need a firewall — paulnovosad · 2026-10-01