Narayanan & Kapoor: p(doom) estimates are still too unreliable to inform AI policy
mikeflache · x · 2026-10-07
Princeton researchers Arvind Narayanan and Sayash Kapoor republish their critique of quantitative AI existential-risk estimates: p(doom) figures lack inductive, deductive, or track-record foundations and effectively launder vague intuitions through pseudo-quantification.
Key points:
- Calculating precise probabilities for unprecedented catastrophes is mathematically infeasible; these numbers don't come from validated models;
- Basing public policy on speculative, biased guesses risks misguided, restrictive regulation;
- The critique targets AI x-risk forecasting specifically, not forecasting in general;
- They urge policymakers toward flexible frameworks addressing concrete, observable AI harms instead.
More from AGI Musings
- Turing laureate Patterson: AI will replace all jobs by end of 2030, knowledge work to drop below 1% of today's price — davidpattersonx · 2026-10-07
- Blogger Slams Anthropic's Effective Altruism Culture: Ideology Isn't Humanity's Consent — VraserX · 2026-10-07
- Grady Booch: a non-small number of mathematicians are deep in soul-searching — Grady_Booch · 2026-10-07
- eigenrobot: ethical norms work like evolved organisms, not rational machines — eigenrobot · 2026-10-07
- The Model Is Not the Mind: Rethinking Agent Economics via Erlang/BEAM — arthurcolle · 2026-10-07
- Average Math Result Took 3 Hours of ChatGPT Pro Compute, Sparking Scaling Debate — birchlse · 2026-10-07