Ben Goertzel: which minds have beneficial 'basic AI drives'? Only non-maximizers
bengoertzel · x · 2026-10-19
Ben Goertzel publishes a new essay responding to Steve Omohundro's 2008 'Basic AI Drives' argument that nearly any advanced AI will self-improve, guard its goals, resist shutdown and grab resources. Goertzel argues this only holds for single-score maximizers with frozen goals, and walks through Omohundro-style reasoning for a different design: many objectives, firm ethical limits, doubt-tempered values that regrow when damaged. TL;DR: minds designed from the start with beneficial, self-stabilizing goal systems.
Related event: Ben Goertzel asks which superintelligent minds develop benign AI drives(3 posts)→
More from AGI Musings
- Robert Greene's advice to young writers: "don't touch AI, don't even go near it" — david_perell · 2026-10-20
- Peter Diamandis: A generation loses confidence in the future before it loses every job — PeterDiamandis · 2026-10-20
- Answers are commoditized, so the quality of your questions is now the top signal — signulll · 2026-10-20
- AI exposes creative industry's uncomfortable truth: much 'talent' was just access — AIandDesign · 2026-10-20
- "AI is a medium that turns labor into capital" — a take going around — curious_vii · 2026-10-19
- AI today is the slowest you'll ever use — in 5 years it'll look like dial-up, says founder — cramforce · 2026-10-19