jessi_cata on cosmic Darwinism, singletons, and why effective agents might not need a singleton
jessi_cata · x · 2026-11-18
AI alignment discussant jessicata lays out a framework on effectiveness vs. consumption of terminal goals: some means are more effective at replication or convergent instrumental goals, and one can imagine a state of nature with many effective agents where the more effective wins resource conflicts — an alternative to Bostrom's singleton that performs cosmic (eu/dys)genics to preserve room for "hobbyist" fun over pure productivity.
The thread traces a lineage from Friendly AI → AI safety → anti-successionism → anti-handoverism, and an intention to preserve multipolarity: the author doesn't especially like singletons and is fairly okay with cosmic Darwinism, noting it's quite different from what Friendly AI was supposed to be.
More from AGI Musings
- Anthropic researcher says AI will make nearly all humans permanently unemployable; swyx pushes back on the economic paradox — khademinori · 2026-11-18
- lateinteraction questions whether AI is rebuilding the economy from scratch with extra steps — lateinteraction · 2026-11-18
- Alignment researcher davidad cuts P(doom) from 70% to 5-8%, citing Claude 3 Opus — repligate · 2026-11-18
- "AI for epistemics" is disconnected from reality, argues researcher — zetalyrae · 2026-11-18
- Three iron laws of AI safety: no one wrote it, coordinated it, or will do it instead — AndyMasley · 2026-11-18
- The 99/1 rule: HBR/BCG study finds heavy AI oversight brings 33% more decision fatigue, 39% more errors — sebpaquet · 2026-11-18