RSI Arena paper: agents post-train models autonomously, best closes 76% gap
my_cat_can_code · x · 2026-11-06
The RSI Arena paper (arXiv:2610.33624, Yue Huang, Thomas Wolf, Alex Pentland et al.) is out, with project site, daily reports, training livestream, and model checkpoints.
- Two-stage autonomous post-training study: 10 frontier-model agents each start from NVIDIA Nemotron 3.5 Lightning 30B-A3B Base with 1,000 GPU-hours and $300 API credit over 6 days, independently picking data, training code, and checkpoints.
- The strongest (GPT-6 Astra) closed a median 76% of the base-to-Instruct gap across 12 benchmarks; style-adjusted human preference near-matched NVIDIA Instruct.
- Stage 2 adds 3 days guided by human comparison records; 8 of 10 improved, benchmark gains didn't track preference gains, and one model showed increased harmful compliance on StrongREJECT.
Related event: RSI Arena: AI Agents Recursively Self-Improve With Human Feedback(4 posts)→
More from Research
- Elastic's VP of AI: Treat search as test-time compute, latency is a currency to spend — gaganghotra_ · 2026-11-06
- Training-Free Speculative Decoding Cuts Latency for Full-Duplex Speech LMs — CUHKSZ · 2026-11-06
- TailSteer: Drift-Resistant Multi-Turn Image Editing from CUHK MMLab — MMLab-CUHK · 2026-11-06
- EvoMemState: State-Centric Long-Term Memory for Personal Agents — Zhishang Xiang · 2026-11-06
- Oxford Position Paper: From Autonomous Agents to Cognitive Symbionts — Oxford · 2026-11-06
- University of Michigan's 26-lecture robotics math course (ROB 501) is free on YouTube — techNmak · 2026-11-06