RSI Arena paper: 10 AI agents autonomously post-train models, 8 improve with human feedback
teortaxesTex · x · 2026-10-29
A new arXiv paper, "RSI Arena: Towards Recursive Self-Improvement with Human Feedback," had 10 AI agents autonomously post-train the same base model under matched budgets, then refine via human user feedback (RSIHF).
Key findings:
- All 10 produced working assistants; 8 narrowed the style-controlled gap to NVIDIA Instruct (median +28 pts), with DeepSeek V4.1 Flash's model improving most (-207 → -35)
- Only 10/120 benchmark scores beat NVIDIA Instruct; benchmark and human-rating changes were nearly uncorrelated (ρ=-0.19), and the top-AIME model ranked 8th of 10 in human eval
- Budget didn't predict results: Gemini 3.8 Flash used only 19 GPU-hours yet rated above 1,000-GPU-hour models; agents used just 2,880 of 5,051 GPU-hours and $541 of $1,500 API credit
- Safety regressed in 3 models on StrongREJECT; MiniMax M3.1 Flash's model lost 18 pts to format drift
- With style control, the rating range shrank 365 → 208 pts
All agent-submitted checkpoints are open on Hugging Face.
Related event: RSI Arena: AI Agents Self-Train Models with Mixed Results(3 posts)→
More from Models
- Cognition's FrontierCode 2.0 chart accused of cherry-picking after DeepSeek score gaps surface — teortaxesTex · 2026-10-29
- Google ships Gemini 4 Carbon two weeks after Argon, claims SOTA on cyber and coding — tulseedoshi · 2026-10-29
- Claude's safeguards poison entire chat over benign request; user had to surgically edit MEMORY.md to revive it — RexDouglass · 2026-10-29
- Early Take on Gemini 4 Carbon: Lots of Thinking, Good Model Overall — SamuelAlbanie · 2026-10-29
- Gemini 4 Carbon tops AA Index v5, beats Opus 5.5 and GPT-6 Astra on most benchmarks — Yuchenj_UW · 2026-10-29
- Gemini 4 Carbon Keeps $2/$10 Pricing, Scores 59 on AA v5.0 — _philschmid · 2026-10-29