Metzger: If Someone Gives You an AI Extinction Probability, Ask for the Math
nptacek · x · 2026-09-17
Perry Metzger proposes a simple test: whenever someone claims AI has an X% chance of wiping out humanity, ask them for the calculation that produced that number. If they never did a calculation and simply pulled the figure out of thin air, you can treat it like anything else that came from the same place.
It is a direct challenge to the habit of quoting precise-sounding existential-risk numbers without any underlying quantitative work — part of the ongoing methodological debate inside the AI safety community.
More from AGI Musings
- UW professors on the 'grand challenge' of building transparent, beneficial AI — lazowska · 2026-09-17
- Get past the next 20 years and AI could usher in a golden age — Objective_Singer_404 · 2026-09-17
- AI is killing cold email: professor gets 75 PhD inquiries months before Fall 2027 cycle — xuanalogue · 2026-09-17
- 25 Fields Medalists Sign Declaration Warning Against Measuring AI Progress by Solved Problems — krishnan · 2026-09-17
- How Effective Altruism Lost Its Way: Matt Johnson's 2023 Essay Still Holds Up — inductionheads · 2026-09-17
- UW researchers push back on AI doom fears: the real risk is unaudited agent systems — lazowska · 2026-09-17