Can each model generation fixing the last one avert xrisk? Researchers clash
krishnanrohit · x · 2026-10-13
Rohit Krishnan argues that gradual capability growth across generations is sufficient: if model gen n+1 mitigates gen n's problems even lossily, xrisk scenarios become extremely implausible. David Manheim disagrees: this bets that systems won't get smarter and better at deception faster than companies' mitigations can prevent smaller failures — a bad bet given short-term incentives. He adds that current safety research addresses only one problem and won't fix it for stronger models.
Related event: Alignment Debate: Can Commercial Incentives Keep Powerful Models Supervised(8 posts)→
More from AGI Musings
- Mistral CEO Arthur Mensch: Europe must build frontier AI or surrender — IsForAt · 2026-10-16
- LukeW notes a spike in Physics majors, crediting the AI wave — LukeW · 2026-10-16
- Agents will use your app however they want — computer use makes control naive — andreisavu · 2026-10-16
- Developer slams AI safety discourse: feeding LLM output back isn't nukes — nitarshan · 2026-10-16
- AI dilutes everything painstakingly created — "enjoy the honeymoon" — sull · 2026-10-16
- 'Refusing to code with AI in 2026 is like refusing Google in 2005' — prasenx · 2026-10-16