Muon may help agentic RL post-training
Kai Ruan · hf · 2026-07-20
Muon can improve agentic RL post-training
The paper studies whether Muon helps sparse-reward agentic reinforcement learning, where its benefit was previously unclear.
- On ALFWorld with Qwen2.5-0.5B-Instruct, the authors compare vanilla Muon against AdamW under matched single-seed runs.
- With GiGPO, applying Muon only to hidden weight matrices raises final-window validation success from 0.290 to 0.546 (+88%). High-rate AdamW controls do not retain post-update success.
- The effect depends on both the advantage estimator and the learning rate.
- At 3e-5, Muon improves GRPO from 0.161 to 0.268.
- For GraphGPO, the gap narrows near saturation at 1e-5, while Muon reaches 0.901, improves normalized validation AUC from 0.399 to 0.556, and hits 0.5 and 0.75 success 30 and 60 updates earlier.
The authors conclude that Muon can help agentic RL, but multi-seed and cross-task validation are still needed.
More from coding & agent
- GitHub Copilot team routes user bug reports to an AI agent via Slack — marlene_zw · 2026-09-11
- Scanning 23 agent sessions, a dev found 3 silent failure modes in memory systems — No_Advertising2536 · 2026-09-11
- eslint-plugin-react v8.0.2 adds 4 checks for React 19.3, supports ESLint 10 and Biome — viglovikov · 2026-09-11
- Arkon: open-source MCP server turns enterprise SOPs into a traceable LLM knowledge wiki — tom_doerr · 2026-09-11
- Cheaper OpenAI Agents API alternative: sandbox service undercutting E2B by 46% — airesearch12 · 2026-09-11
- His agent kill switch ran for months before he found it was wired to nothing — AnvilandCode · 2026-09-11