Don't panic about RSI: we can still study the code AI writes
patpat_mit · x · 2026-09-22
- Prof. Noah Gian pushes back on the idea that recursive self-improvement (RSI) means we'll stop understanding AI.
- Two arguments: (1) we already don't understand AI behavior despite knowing how they're designed; (2) even with RSI, a model is software written in a programming language, and LLMs are great at explaining code to us.
- He concedes RSI could accelerate hyperparameter optimization, design innovations, and learning paradigms, but argues we can keep studying the code whether AI writes 0%, 80%, or 100% of it. Invites counters: "Am I missing something?"
More from AGI Musings
- Reality is moving at 70-90% the speed AI 2027 predicted, re-evaluation finds — ben_j_todd · 2026-09-22
- AI Risk Researcher: My Concern Stems From Ignorance, Not Prescience — danfaggella · 2026-09-22
- Google CISO Phil Venables: Defense Against AI Attacks Needs Autonomy, Not Just Speed — philvenables · 2026-09-22
- AI in GTM Will Decimate Teams That Were 'Tolerated, But Not Liked' — thedealdirector · 2026-09-22
- Mathematician littmath responds to mob demanding immediate release of isomonodromy work — littmath · 2026-09-22
- AI researcher: math results should be open, but a few months' delay is fine — littmath · 2026-09-22