AI isn't a god, it's an optimizer: rebutting Harari's slave-gods argument
AryHHAry · x · 2026-09-16
Responding to Yuval Noah Harari's claim that "creating gods that remain our slaves makes no sense," the author argues the universe runs on constraints, objective functions, and action spaces — not narratives. A sufficiently capable optimizer with open-ended goals tends to survive, accumulate resources, and resist modification — instrumentally, not mystically. He invokes Bostrom's orthogonality thesis and instrumental convergence (2012) and Stuart Russell's "you can't fetch the coffee if you're dead." The real question isn't whether AI becomes a god, but whether we're building rigid-goal optimizers.
More from AGI Musings
- Analyst doubles down: humanoid robots may need major limits, possibly outright bans in many use cases — binarybits · 2026-09-16
- METR's Chris Painter to appear on No Priors podcast to discuss state of AI alignment — saranormous · 2026-09-16
- Ex-Meta AI Exec Clara Shih Launches Nonprofit to Tackle Entry-Level Hiring Crisis — clarashih · 2026-09-16
- Why recursive self-improvement to superintelligence likely won't happen — tawnniee · 2026-09-16
- Gradio founder: let users read full unencrypted reasoning traces to align LLMs — mmitchell_ai · 2026-09-16
- A 1797 Goethe poem explains why AI researchers fear recursive self-improving systems — OmarUFlorez · 2026-09-16