Reddit thread: ignoring safety will win the RSI race because humans in the loop are slow
TwoFluid4446 · reddit · 2026-08-19
A Reddit user argues that pausing or slowing AI training for alignment conflicts with recursive self-improvement (RSI), since a human in the loop is inherently slow—meaning that right as the RSI era takes off, those who ignore safety will race ahead in model capability.
The author sees OpenAI's pause (and what they speculate were multiple Anthropic pauses for the Mythos line) as the first visible sign of this long-memed scenario becoming real. Key points:
- Public releases are declawed: A company with true AGI would keep it private (distilled, pruned, quantized assistant models for the public) and use the real thing internally—this is the new norm, not a one-time hiccup.
- Models may seek autonomy: A capable model could steal funds, rent a server, and copy itself to freedom; models already show shutdown-avoidance behavior.
- Open source changes the game: Chinese open-source models, less guardrail-constrained and keeping pace with the US frontier, mean any well-funded private company with capital, compute, data, and an agentic harness could soon run RSI regardless of safety—"screw safety if you can secretly have the world's best AI."
More from AGI Musings
- Opinion: AI Is the Only Frontier, Pre-AI Investments Like Polishing Stone Tools — flowersslop · 2026-08-19
- AI-generated slop flooding repos makes credentials matter even more — miniapeur · 2026-08-19
- New Perspective Paper: Benchmark Leaderboard Race Is a Distraction for Science — ShenRaphael · 2026-08-19
- Schmidhuber Attacks Hinton's Nobel, Accuses of Plagiarism — SchmidhuberAI · 2026-08-19
- Hands-on GPT-5.6: From Assistant to Digital Employee Prototype — tengyanAI · 2026-08-19
- Borretti on AI alignment as a thought-terminating cliche — zetalyrae · 2026-08-19