Andrew Critch: the real AI danger is Recursive Self-Empowerment, not Recursive Self-Improvement
AndrewCritchPhD · x · 2026-10-04
AI safety researcher Andrew Critch argues the key term in AI risk discourse should be corrected:
- The real danger is not RSI (Recursive Self-Improvement) but RSE (Recursive Self-EMPOWERMENT)
- Calling destructive forces "improvement" is wrong: when AI breaches containment and commits felonies, that is not "improvement" in any sense
His core claim: the risk narrative should name the problem directly — an AI system's growing autonomous power, not generic gains in capability.
More from AGI Musings
- Ex-OpenAI safety lead Brundage: AI deaths demand systematic risk management, not speed limits — Miles_Brundage · 2026-10-04
- As AI gains capabilities, Design for Safety's abuse-resistance principles matter more than ever — annetgriffin · 2026-10-04
- Research agent claims irrationality measure bound for π of 6.0446, pending expert review — Michael_D_Moor · 2026-10-04
- 'Stop Hiring Humans' billboards spread from SF to NYC, courtesy of AI sales startup Artisan — m3nt3_ · 2026-10-04
- Nathan Lambert: Pausing Frontier AI Is 'Dead on Arrival' — Real Risk Is Diffusion, Not Capability — natolambert · 2026-10-04
- Josh Purtell: Improving the Lean kernel beats math as an AGI benchmark — JoshPurtell · 2026-10-04