Survey of AI researchers puts extinction-risk estimates near 18%, sparking debate

On September 15, 2024, the longest-running and largest survey of AI researchers released its latest round: 1,580 researchers who published at top ML/AI venues were asked about catastrophic/extinction risk from AI. Respondents gave a mean 18% probability of human extinction or similarly irreversible disempowerment—nearly one in five. The figure has fueled ongoing debate over survey methodology and media reporting, making it a centerpiece of public AI-risk discussion.

Confirmed

• Results were released as a paper (Nathan Young among the authors); David Krueger and Nathan Young both amplified it.
• Mean estimate 18%; typical/median estimate 10%, up from the previous round, with 10% being a common, normal estimate among researchers; many respondents ranked far higher. gleech cited another ML-researcher survey with a median extinction-risk estimate of 10% and 81% assigning risk above 1%, and corrected an earlier figure (from "58% at ≥5%" to 81% in his "doomer" range).
• gleech listed 70+ years of AI-risk warnings from Turing (1951) through Wiener, Good, Vinge, Bill Joy, and Hawking, using the survey to support that lineage.
• Katja Grace's chart shows the concern distribution shifted upward in one year; timelines to human-level AI shortened sharply—a compounding trend.
• Asian researchers reported higher concern than Western ones, contradicting the "China will race ahead" assumption; Grace questioned why US slowdown arguments assume asymmetric risk attitudes.
• DrTechlash noted the overlooked flip side: respondents averaged 52% probability that HLMI's overall impact is good, with only 28% bad outcomes.

Unconfirmed

• DrTechlash's methodological criticisms: vague wording (extinction bundled with undefined "disempowerment") and only 8% effective response rate warrant caution in interpreting exact numbers.
• NathanpmYoung and Melanie Mitchell are debating the widely circulated claim that "half of AI researchers put extinction risk at ≥10%"; Mitchell's fact-check found errors in NYT, Vox, and New Yorker coverage.
• Nathan Young, citing co-author Katja's FAQ, raised whether percentages are even a useful way to express AI risk.

Why it matters

• Grace reports two points of consensus from field conversations: AI progress has exceeded expectations, and concern about catastrophic risk keeps deepening.
• Nathan Young argues the survey is among the best primary data on experts' true probability distribution over AGI risk.
• The methodology-and-media controversy shows that accurately representing expert risk judgment is itself now a key issue in the public AI-risk debate.

2026-09-15 ~ 2026-09-17 · 17 related posts

• [source] 2024 survey: 10% AI extinction risk is normal among 1,580 AI researchers — NathanpmYoung · 2026-09-15
• AI Researchers Put ~18% Probability on AI Causing Human Extinction, Survey Finds — NathanpmYoung · 2026-09-15
• The researcher behind the largest peer-reviewed survey of AI researchers — KatjaGrace · 2026-09-15
• Largest long-running survey of AI researchers tracks shifting expert opinion since 2016 — KatjaGrace · 2026-09-15
• Expert surveys show AI timelines shrinking fast and extinction concerns rising — KatjaGrace · 2026-09-15
• [source] Largest AI researcher survey charts rising concern about catastrophic risk over one year — KatjaGrace · 2026-09-15
• Survey: Asian AI researchers worry more about AI risks than Western peers — KatjaGrace · 2026-09-15
• Landmark AI researcher survey: most experts see 10% chance AI destroys humanity — DavidSKrueger · 2026-09-15
• Debate Rekindled: Do Half of AI Researchers Really See 10% Extinction Risk? — NathanpmYoung · 2026-09-15
• AI extinction risk estimate rises to 10% in expert survey; debate over probability framing — NathanpmYoung · 2026-09-15
• AI safety researcher revises doomer estimate to 81% in ongoing risk survey debate — gleech · 2026-09-15
• [source] AI Impacts survey critics flag vague wording, tiny effective sample and MIRI ties — DrTechlash · 2026-09-15
• Same AI Impacts survey: participants average 52% on HLMI being net positive — DrTechlash · 2026-09-15
• From Turing to Hinton: a 70-year lineage of AI risk warnings — gleech · 2026-09-15
• Survey: ML researchers' median xrisk guess is 10%, 81% put >1% on it — gleech · 2026-09-15
• Median AI existential risk estimate hits 10%; pretraining down to 11% of compute — gleech · 2026-09-17

1 near-duplicate retellings: NathanpmYoung