Long-horizon agent RL recipe: Muon optimizer with single-rollout async training boosts repo-level issue resolution
Kai Ruan · hf · 2026-10-15
Researchers from Renmin University, Duke, and CAS examine RL for long-horizon agents, where sparse outcome rewards require weighting trajectories of different lengths and updating policies while rollouts are still collected.
- Proposes a single-rollout asynchronous recipe: Muon optimizer + token-averaged loss + truncated importance sampling + auxiliary length and tool-error penalties.
- Single-rollout sampling enables asynchronous learning without waiting for multiple trajectories per issue.
- Across two model families on repository-level issue resolution, the full recipe achieves higher training success than baselines.
More from coding & agent
- Player-1: RL turns VLM agents into GTA V players with zero game-specific training — yshan2u · 2026-10-15
- Agentic engineering vs AI engineering: one builds software with agents, the other builds agents as the product — Al_Grigor · 2026-10-15
- Cursor + Gemini 4 Argon builds a multiplayer 3D capture-the-flag game in one go — FinanceYF5 · 2026-10-15
- Claude Opus 5.5 levels up codegen but its docs and PR prose are 'unusable' — indragie · 2026-10-15
- Gary Marcus flags AI agent fatigue: managing coding agents burns out engineers — GaryMarcus · 2026-10-15
- The thrill of building isn't gone — you're just not stopping to enjoy it — _jaydeepkarale · 2026-10-15