Researchers push back on the claim that AI has a 10% chance of ending humanity within a decade

A two-day intense exchange among researchers centered on the claim that "AI has a 10% chance of ending humanity within the next decade," with core disagreements over how the proposition is defined and whether realistic extinction pathways actually exist.

Confirmed

• mbateman posted repeatedly to rebut the claim: literally having AI completely exterminate humanity is extremely difficult, and the strongest scenarios imaginable—shoggoth, grey goo, paperclip maximization—fall far short of supporting a "10% in the near term" probability assertion.
• mbateman further criticized how the proposition is phrased: he argued that "AI has a 10% chance of ending humanity within the next decade" is false in a way that makes it nearly impossible to reason about, whereas "there is a 10% chance someone causes enormous damage with AI within the next decade" is more plausible and at least tractable.
• In the biorisk discussion, magnushambleton argued that judgments about biological risk depend almost entirely on whether one has engaged deeply with biologists—for a motivated biological superintelligence, exterminating a species would be nearly trivial; the key variable is the superintelligence, not the bioweapon itself.
• AI safety researcher David Manheim clarified his position: he does not consider AI×bio risk to be low, but points out that "AI designing a completely new supervirus beyond existing viruses" is unrealistic before a misaligned ASI emerges.
• Researcher Afinetheorem offered a conditional reading of high-probability claims: under the assumption of "no development constraints," the probability is indeed above 10%, but by the same standard, nuclear weapons and bioweapons "being able to kill all humans" should also hold; conditioned on the median development path, the actual probability is far lower.

Why it matters

• This debate shows that disagreements over AI p(doom) are often not disagreements about facts, but about proposition definitions, time horizons, and conditioning assumptions: unconditional probabilities differ vastly from median-path probabilities.
• The exchange between Manheim and Hambleton further distinguishes two types of risk: AI-assisted bioweapon misuse could happen today, while "designing a completely new supervirus" presupposes a misaligned superintelligence—the two should not be conflated.

2026-09-09 ~ 2026-09-11 · 6 related posts

• [source] "10% AI Ends Humanity" Is Unreasonable; "Someone Uses AI for Damage" Is the Real Question — mbateman · 2026-09-09
• [source] Blogger pushes back on '10% AI extinction in a decade': no plausible path exists — mbateman · 2026-09-09
• The Real Variable in AI X-Risk Isn't Bio, It's Superintelligence, Argues Bateman — mbateman · 2026-09-10
• [source] Manheim: AI-Designed Superviruses Aren't Realistic Before Misaligned ASI — davidmanheim · 2026-09-10
• Researcher pushes back on AI extinction odds: conditional vs median-path estimates diverge — Afinetheorem · 2026-09-10
• "10% chance AI kills everyone" is nonsense, argues exit-risk skeptic: more like a meteor strike — DeryaTR_ · 2026-09-11