Jon Mellon challenges RSI emergency framing: the risk is speed, not loss of control
jon_mellon · x · 2026-09-21
In a debate with @binarybits, @davidshor and @chrislhayes, Jon Mellon lays out his objection to AI risk narratives:
- What he rejects is the intellectual framework claiming labs are about to trigger RSI (recursive self-improvement) leading to superintelligence, loss of human control, and thus an emergency.
- However, he concedes that even without that framing, RSI remains dangerous in the sense that we may get and entrench technology that is destabilizing in hard-to-predict ways much faster than expected.
A substantive exchange on how doomers and non-doomers frame AI risk differently.
Related event: Debate Over AI Risk Narratives: Doomers vs. Mainstream Safety Agendas(4 posts)→
More from AGI Musings
- 'We Must Regulate the Hurricane!' Satire Skewers AI Governance Debates — Pseudomanifold · 2026-09-21
- Andy Masley Launches Series Explaining Effective Altruism's 'Story of the World' Without Jargon — AndyMasley · 2026-09-21
- eigenrobot: Deontology Liberates Rather Than Burdens, and Shrimp Consequentialism Is Post Hoc Scrupulosity — eigenrobot · 2026-09-21
- Tyler Cowen makes the case for what happens if we choose to slow down AI — paulnovosad · 2026-09-21
- Stratechery: Frontier Labs' Push to Slow AI May Buy Time to Reduce Capability Overhangs — viksit · 2026-09-21
- The core flaw in most AI edtech products: learning outcomes are never defined — _akpiper · 2026-09-21