Ben Goertzel on superintelligent 'Basic AI Drives': what matters is which minds we seed
bengoertzel · x · 2026-10-19
Ben Goertzel (founder of SingularityNET) revisits Steve Omohundro's 2008 'Basic AI Drives' paper, which argued almost any advanced AI will tend to self-improve, guard its goals, and protect itself (including against being switched off). His TL;DR: the minds that emerge beneficial will be those designed from the start with beneficial and self-stabilizing goal-systems. The real question is not which superintelligent mind attractors are possible — there are many — but which kind we are collectively motivated to seed.
Related event: Ben Goertzel asks which superintelligent minds develop benign AI drives(3 posts)→
More from AGI Musings
- "It's still OK to enjoy programming": dev pushes back on AI-era coding doom — facontidavide · 2026-10-20
- 26,811-pupil study: AI lifts homework scores in China but hurts exam performance — Dr_Alex_Crimi · 2026-10-20
- Philosopher Negarestani answers 'The Superdark Factory': operational agency is outpacing rational integration — tylerjdunn · 2026-10-20
- AI gives us access to 'Metis' — the practical wisdom Scott said the state destroyed — StewartalsopIII · 2026-10-20
- Stanford HAI's 2026 AI Index: AI capabilities surge while governance frameworks lag behind — StanfordHAI · 2026-10-20
- Why Systems Neuroscientists Should Stay Quiet on NeuroAI: A Philosophical Case — KordingLab · 2026-10-20