Muon nearly doubles agentic RL success in GiGPO, but only with the right setup

rohanpaul_ai · x · 2026-07-22

Muon can substantially improve agentic reinforcement learning, but only under the right training setup.

• The paper compares vanilla Muon against AdamW on sparse-reward agentic RL using matched single-seed comparisons.
• With GiGPO, which estimates advantage from repeated states, Muon raises late-window validation success from 0.290 to 0.546 — an 88%+ relative gain.
• The authors find the effect depends on both the advantage estimator and learning rate: at 3e-5, Muon improves GRPO from 0.161 to 0.268; at 1e-5, the gap narrows as GraphGPO’s late-window performance nearly saturates.
• They conclude that an optimizer should not be judged in isolation; the surrounding RL setup can determine whether it helps or hurts.

Original post →

More from Research

• LightMTP: lightweight latent multi-token prediction boosts LLM planning in pretraining — gdm3000 · 2026-10-27
• EMNLP study: AI literary translation reads fine but varies more, even within one book — tuvllms · 2026-10-27
• Lab monkeys at $30-80K each: a 4-year breeding cycle is the real preclinical drug bottleneck — zebird0 · 2026-10-27
• OpenAI Foundation funds UNC tumor antigen project, hiring senior experimental scientist — iskander · 2026-10-27
• Steering attention heads toward honesty: up moral patienthood, down psychopathy — edelwax · 2026-10-27
• Are transcoders overkill? New paper argues plain neurons suffice for circuit analysis — burny_tech · 2026-10-27

Research channel →