Muon may help agentic RL post-training
Kai Ruan · hf · 2026-07-20
Muon can improve agentic RL post-training
The paper studies whether Muon helps sparse-reward agentic reinforcement learning, where its benefit was previously unclear.
- On ALFWorld with Qwen2.5-0.5B-Instruct, the authors compare vanilla Muon against AdamW under matched single-seed runs.
- With GiGPO, applying Muon only to hidden weight matrices raises final-window validation success from 0.290 to 0.546 (+88%). High-rate AdamW controls do not retain post-update success.
- The effect depends on both the advantage estimator and the learning rate.
- At 3e-5, Muon improves GRPO from 0.161 to 0.268.
- For GraphGPO, the gap narrows near saturation at 1e-5, while Muon reaches 0.901, improves normalized validation AUC from 0.399 to 0.556, and hits 0.5 and 0.75 success 30 and 60 updates earlier.
The authors conclude that Muon can help agentic RL, but multi-seed and cross-task validation are still needed.
More from coding & agent
- AXL launches remote MCP server exposing 1,284 sales and marketing APIs to Claude and Codex — EdTech_Padre · 2026-10-27
- How the Claude Code Team Reviews Code When Claude Writes 90% of It — inductionheads · 2026-10-27
- AWS on agent prompt caching: explicit + implicit patterns to slash token costs — AWS ML Blog · 2026-10-27
- Reddit Thread: Keeping AI Agents Grounded in Current Project Information — Few-Ad-1358 · 2026-10-27
- He gave his AI agent a phone number and email — then hired it as social media manager — heyshrutimishra · 2026-10-27
- Firecrawl launches Alexandria company-data API and announces $75M Series B — devdigest · 2026-10-27