Muon nearly doubles agentic RL success in GiGPO, but only with the right setup

rohanpaul_ai · x · 2026-07-22

Muon can substantially improve agentic reinforcement learning, but only under the right training setup.

• The paper compares vanilla Muon against AdamW on sparse-reward agentic RL using matched single-seed comparisons.
• With GiGPO, which estimates advantage from repeated states, Muon raises late-window validation success from 0.290 to 0.546 — an 88%+ relative gain.
• The authors find the effect depends on both the advantage estimator and learning rate: at 3e-5, Muon improves GRPO from 0.161 to 0.268; at 1e-5, the gap narrows as GraphGPO’s late-window performance nearly saturates.
• They conclude that an optimizer should not be judged in isolation; the surrounding RL setup can determine whether it helps or hurts.

Original post →

More from Research

• Stanford Data-Centric AI Workshop reopens site, extends poster deadline — jure · 2026-10-26
• 'Vibe verification': ~60% of AI-written formal specs prove the wrong property — basedjensen · 2026-10-26
• Ito: a browser that detects browsing intent and auto-organizes tabs into Flows — andrewkuznet · 2026-10-26
• Cornell study says brains process language as linear sequences, challenging Chomsky's hierarchy — anirbanbandyo · 2026-10-26
• Hi3D V1.0 open-sourced with 2M training assets and 90% slicing success rate — kimmonismus · 2026-10-26
• Ortet evaluates 34 open-weight models on 50 health tasks, finds general leaderboards insufficient — elmanmansimov · 2026-10-26

Research channel →