Ben Goertzel's toy simulations probe the geometry of superintelligence attractor basins
bengoertzel · x · 2026-10-28
Ben Goertzel published the fifth post in his blog series, using deliberately crude toy simulations to quantitatively explore the geometry of "superintelligence space" — the dynamics of goal-sets under self-modification and intelligence increase.
- Targets: attractor basins around kinds of superminds including Omohundro-style selfish utility maximizers, wireheads, and the beneficial compassionate agents he tries to build.
- Core question: how wide and deep are these mind-attractor basins, and what holds a mind in one as capability grows?
- Preliminary takeaways: basin size depends on where systems start, so there is no single number; for AI, starting points are decided by what humans build. The ground between basins tends to slope toward the selfish ASI.
- Goertzel stresses these are evocative toy results, not empirical predictions, but argues more realistic versions could run even without actual superintelligences available.
More from AGI Musings
- GEO: the KDD-2024-defined practice of writing content to be cited by AI answers — robertonappe · 2026-10-28
- AI skeptic's job-destruction worry overlooks countless newly solvable problems — robleclerc · 2026-10-28
- Grant agencies flooded with applications — and admins haven't connected the dots to AI — wandedob · 2026-10-28
- Frontier Models as CapEx, Open Models as OpEx: Where AI Token Spend Is Heading — curious_vii · 2026-10-28
- Chris Albon's thought experiment: how would AI history differ if another community found scaling? — chrisalbon · 2026-10-28
- AAAI/ICLR Reviewer: About Half of Reviews Look AI-Generated, Peer Review in Crisis — dsmirza · 2026-10-28