Ex-Anthropic safety researcher: racing to RSI is hubris, not a prisoner's dilemma
dgrobinson · x · 2026-09-28
Nathan Calvin argues the race to recursive self-improvement may not be a prisoner's dilemma at all — it may simply be the insane, hubristic move it appears to normal people, making unilateral restraint rational and moral. The bar for believing your work could cause mass casualties or extinction yet proceeding anyway should be very high, and he says Anthropic hasn't met it, noting Cold War arms races weren't individually rational either.
More from AGI Musings
- "System 2 models built the brain, but System 1 is building the nervous system" — ai · 2026-09-28
- The Vanishing Apprentice: How AI Is Reshaping the Junior Developer Role — ArtificialOther · 2026-09-28
- AI researcher reassures family: over 90% chance the tech leaves humanity alone — rao2z · 2026-09-28
- Flying supercar, persistent agent, 750 TPS: a wild week of AI launches ahead — ChrisGPT · 2026-09-28
- Reddit post argues LLMs are "a billion narrow AIs stitched together," far from AGI — u_are_mad · 2026-09-28
- Fermi estimate: brain may pack 500k-5M molecular switching units per 'parameter' — JosephJacks_ · 2026-09-28