Metzger: If Someone Gives You an AI Extinction Probability, Ask for the Math
nptacek · x · 2026-09-17
Perry Metzger proposes a simple test: whenever someone claims AI has an X% chance of wiping out humanity, ask them for the calculation that produced that number. If they never did a calculation and simply pulled the figure out of thin air, you can treat it like anything else that came from the same place.
It is a direct challenge to the habit of quoting precise-sounding existential-risk numbers without any underlying quantitative work — part of the ongoing methodological debate inside the AI safety community.
Related event: Metzger Challenges Claims of 10% AI Extinction Risk Within a Decade(2 posts)→
More from AGI Musings
- Investor: in the LLM era 'did it work' replaces diplomas and test scores — StewartalsopIII · 2026-11-04
- Should AGI labs have a reverse Conway's Law — design the org from the artifact? — akbirthko · 2026-11-04
- Verification Is Not Knowledge: Science needs more than AI answers — ersinesen · 2026-11-04
- Andreessen: Agents Are Entering Enterprises as Chaos, Just Like PCs and Spreadsheets Did — Dan_Jeffries1 · 2026-11-04
- 1000+ mathematicians sign pledge to shun AI companies and models — Shesawthat · 2026-11-04
- Von Neumann meme: "You don't have to be responsible for the world that you're in" — akbirthko · 2026-11-04