Why AI lab researchers who believe in ~10% extinction risk keep working there
sebkrier · x · 2026-09-09
Jasmine Sun maps the three reasons people keep working at AI labs despite believing in 10% extinction risk:
- Techno-determinism: someone will build ASI anyway; better it's me than China/OpenAI.
- Consequentialism: ASI might kill us but might deliver utopia/immortality — a +EV bet.
- Self-interest: fun, money, cool tech with friends; ignoring the macro picture.
Notably, none of these is "hyping risk for marketing" — people believe what they say while compartmentalizing. She argues the public is better served by researchers pushing consensus on specific safety solutions (regulatory or otherwise) rather than vagueposting about extinction.
Quoted Anthropic researcher EvanHub says he earnestly believes AI could kill all humans — >10% within the next decade — and that Anthropic is trying its best but has no clear plan or path to solving superintelligence alignment.
Related event: Why AI Researchers Stay Despite Believing Extinction Risk(3 posts)→
More from AGI Musings
- Musk: robot RAM demand will dwarf everything else, and 100% of superintelligence compute goes to deep space — elonmusk · 2026-10-27
- Anthropic S-1 warns of AI job displacement as Microsoft offers exit to ~27% of staff — _jaydeepkarale · 2026-10-27
- eigenrobot: 200 years of attempts to replace Christianity have each fared worse than the last — eigenrobot · 2026-10-27
- Replit CEO on the 'self-driving company': agents to absorb coordination costs by 2030 — amasad · 2026-10-27
- US senator slams AI industry for proposing 50-death threshold for 'catastrophic incident' — peterwildeford · 2026-10-27
- Replit CEO Says AI Makes All Software De Facto Open Source; Aakash Gupta Asks What Moats Remain — aakashgupta · 2026-10-27