AI singleton debate: jessi_cata argues power concentration beats military race on safety but loses to Gnon
jessi_cata · x · 2026-11-15
In an ongoing debate with RokoMijic, jessicata argues that the concepts of "singleton" and "intrusiveness" are being conflated: any powerful system capable of curbing military competition is itself a form of power concentration, and even if it doesn't feel intrusive, it would necessarily disempower people.
He further contends that the military-competition path actually has less concentration of power than an AI singleton, since power is shared among more actors and could be more popular — but it would also be less effective at defeating "Gnon" (the selector outside the system). A representative slice of the AI-safety debate over power concentration vs. existential risk.
Related event: Debate flares over whether a friendly AI singleton safeguards humanity(2 posts)→
More from AGI Musings
- NVDA beat aggressive FY2026 revenue estimates by 4x, deflating S-curve takes — abhiadesai · 2026-11-15
- tszzl: Activism-shaped problems usually turn out to be technology problems — tszzl · 2026-11-15
- AI safety campaign calls for global ban on superintelligence and recursive self-improvement — marcvanderchijs · 2026-11-15
- Researcher argues boosting pretraining, not RL, is the safer path to AI alignment — maksym_andr · 2026-11-15
- Jacob Coxon, Anthropic's viral resignee, says he might return to an AI lab — rohanpaul_ai · 2026-11-15
- DeepMind researcher: AGI will likely decentralize power, not centralize it — sebkrier · 2026-11-15