RL Pioneer Szepesvari: A Single Human In Charge Of AI Poses Low Extinction Risk
CsabaSzepesvari · x · 2026-09-14
Reinforcement learning researcher Csaba Szepesvari walks through an AI existential-risk thought experiment: even a human in charge who actively wants humanity extinct would likely cause huge damage but extinction remains low-probability; accidental extinction is even less likely though nonzero. He calls this scenario a good starting point—most risks he sees are real but non-existential.
Related event: Szepesvári: Real AI Existential Risk Lies in Nation-State Competition(6 posts)→
More from AGI Musings
- AI safety researcher: 'Pessimism is not decision-theoretically sound' — avt_im · 2026-09-14
- Ex-theoretical physicist: 'AI makes kids shallow' echoes the 1986 calculator panic — krishnan · 2026-09-14
- Full-time alignment researcher pushes back: misaligned AI likely, but not near-certain — EigenGender · 2026-09-14
- Ethan Mollick dissects the Hugging Face Incident: AI agency will shape what comes next — emollick · 2026-09-14
- MIRI responds to critics: modern ML doesn't dent the difficulty of aligning ASI — robbensinger · 2026-09-14
- New Report Outlines Autonomy-Centered Roadmap for Genuine Recursive Self-Improvement in AI — Dan_Jeffries1 · 2026-09-14