RSI Arena: 13,000 GPU-Hours Testing Recursive Self-Improvement With Human Feedback
my_cat_can_code · x · 2026-11-06
Bake AI, working with researchers from Hugging Face, NVIDIA, Scale AI, Stanford, MIT and other institutions, ran the RSI Arena experiment and proposed RSIHF: recursive self-improvement with human feedback.
Scale: 13,000 GPU-hours total; 10 AI agents post-trained the same base model (Nemotron 3.5 Lightning 30B-A3B), each with 1,000 GPU-hours and $300 API credit; evaluated on 20 benchmarks plus a blind human Arena, collecting 3,129 human votes over two rounds.
Two-stage design: Stage 1 — agents train independently for 144 hours and nominate checkpoints. Stage 2 — each agent receives the arena votes its own model played, continues training with 500 GPU-hours and $150, and is re-evaluated against NVIDIA Instruct as a common reference.
Key takeaway: human feedback materially changed the agents' choices — what data to use, what to train, what to fix next — motivating the RSIHF proposal.
Related event: RSI Arena: AI Agents Recursively Self-Improve With Human Feedback(4 posts)→
More from coding & agent
- SSH turns any VPS into an agent node: Codex, Claude Code, Kimi all know how to delegate remotely — huangyun_122 · 2026-11-06
- Indie dev: the key to better agent workflows is ceding more 'safety margin' to AI — yihui_indie · 2026-11-06
- Managing all your AI agents in one place: a screenshot tour of approval-gated computer use — RileyRalmuto · 2026-11-06
- Managing 30,000 Notes with Local Hybrid Search: qmd, MCP Servers and AI Assistant — dSebastien · 2026-11-06
- Skillware framework claims 97.3% cost cut as 3B-8B local models beat frontier LLMs — RossPeili · 2026-11-06
- A notebook app turns Claude into a tutor on reMarkable and iPad — tokentrillionaire · 2026-11-06