Goertzel's 2010 classic: 'The Singularity Institute's Scary Idea, and why I don't buy it'
burny_tech · x · 2026-10-06
While recommending Goertzel's new rebuttal to Yudkowsky, burnytech resurfaces Goertzel's 2010 blog post 'The Singularity Institute's Scary Idea (and Why I Don't Buy It).'
- The 'Scary Idea': anyone actively building advanced AGI who succeeds is highly likely to cause involuntary human extinction.
- Goertzel clarifies he opposes neither SIAI as an institution nor most of its work—only this specific stance held by some of its researchers.
- The post recounts his history with SIAI (advisory roles, Kurzweil's overlapping affiliations) and lays out his systematic objections.
- It's the source text for the decades-long Goertzel–Yudkowsky/MIRI rift.
More from AGI Musings
- Ex-UK energy official warns AI needs 500GW by 2035 and the industry isn't ready — ShakeelHashim · 2026-10-06
- Calling a text sampler a superintelligence will end badly: OOD cliffs and the hype gap — gerardsans · 2026-10-06
- John Langford's COLM 2026 Keynote 'Research at AI Speed' Examines the Industrialization of R&D — JohnCLangford · 2026-10-06
- Nearly 4 years of ChatGPT: zero direct AI deaths vs 225M from disease and aging — DeryaTR_ · 2026-10-06
- 2048 Satire: Every Job Automated Except Healthcare Admin and Unionized Work — rickasaurus · 2026-10-06
- MIT economist: even the most optimistic AI scenario means major labor disruption — soumitrashukla9 · 2026-10-06