Muon vs Shampoo: easier to implement with fewer hyperparameters, but lower ceiling
_arohan_ · x · 2026-11-13
In a reply to arohan, xidulu compares two modern optimizers: Muon is faster to understand, easier to implement, and has fewer hyperparameters to get wrong, while Shampoo has a higher upper bound and potential. A useful practical reference for optimizer selection in training: Muon for fast adoption, Shampoo for max ceiling.
Related event: Muon vs Shampoo optimizer debate reignites in AI community(2 posts)→
More from Research
- OpenKB: open-source LLM knowledge base hits 4.9k stars with vectorless reasoning-based retrieval — tom_doerr · 2026-11-13
- PhysCaP: NTU and NVIDIA's agents measure mass and stiffness with no extra sensors — shaohua0116 · 2026-11-13
- METR time-horizon fit called too pessimistic, redone — with a superexponential meme chart — lxrjl · 2026-11-13
- Meta's Papailiopoulos: Training's Future Is Agents That Compile Intent Into Fully Bespoke FLOPs — DimitrisPapail · 2026-11-13
- Further fine-tuning CLIP with contrastive losses often fails on out-of-distribution data — wightmanr · 2026-11-13
- Interaction-as-a-Judge beats code rewards: +6.7pp win rate, 35.7% less code bloat — arena · 2026-11-13