AI risk debate: tszzl says classic risks won't extinction us, peer points to biorisk
JMannhart · x · 2026-09-11
A debate on existential AI risk: @tszzl argues none of the "respectable" AI risk factors in RSP/preparedness reports (nuclear, pandemics) come close to human extinction — only a grey-goo-like family of self-replicators could. @JMannhart pushes back that he's vastly underestimating biorisk: while accidental catastrophic bio via evolution-like dumb optimization is hard, at cosmic scale even a half-skilled optimization process operating in biology could pull it off. The exchange highlights the core split in AI risk assessment between lab-report frameworks and self-replicating-optimizer scenarios.
Related event: Debate over AI extinction risk: grey goo vs. supply-chain automation(6 posts)→
More from AGI Musings
- Replit CEO on the 'self-driving company': agents to absorb coordination costs by 2030 — amasad · 2026-10-27
- US senator slams AI industry for proposing 50-death threshold for 'catastrophic incident' — peterwildeford · 2026-10-27
- Replit CEO Says AI Makes All Software De Facto Open Source; Aakash Gupta Asks What Moats Remain — aakashgupta · 2026-10-27
- Understanding the AI consciousness debate: which theory of mind is behind it — msharmas · 2026-10-27
- Functionalism: if consciousness is just what the brain computes, silicon can run it too — msharmas · 2026-10-27
- iPhone case with 85,000 reviews in days: everything fake in the age of abundance — tekbog · 2026-10-27