Muon nearly doubles agentic RL success in GiGPO, but only with the right setup

rohanpaul_ai · x · 2026-07-22

Muon can substantially improve agentic reinforcement learning, but only under the right training setup.

• The paper compares vanilla Muon against AdamW on sparse-reward agentic RL using matched single-seed comparisons.
• With GiGPO, which estimates advantage from repeated states, Muon raises late-window validation success from 0.290 to 0.546 — an 88%+ relative gain.
• The authors find the effect depends on both the advantage estimator and learning rate: at 3e-5, Muon improves GRPO from 0.161 to 0.268; at 1e-5, the gap narrows as GraphGPO’s late-window performance nearly saturates.
• They conclude that an optimizer should not be judged in isolation; the surrounding RL setup can determine whether it helps or hurts.

Original post →

More from Research

• New paper: Absolute pose estimation from affine cues and gravity direction — ducha_aiki · 2026-09-11
• LoMa Paper Ships REALLY HardPairs Dataset, Accepted at ECCV 2026 — ducha_aiki · 2026-09-11
• Johns Hopkins Launches Full-Stack Hands-on Robot Learning Class with SO-101 Arm Kits — _krishna_murthy · 2026-09-11
• SyncWorld: In-Context Robot World Model Simulates Unseen Views and Embodiments Zero-Shot — ChongZzZhang · 2026-09-11
• A 3D Pose Dataset for Dogs Released — ducha_aiki · 2026-09-11
• Five tells that still make AI video read as AI, from physics glitches to missing operators — NewPhoneWhotiz · 2026-09-11

Research channel →