Why wasn't Muon tried earlier? Yaroslav Bulatov on optimizer preconditioning mindsets
_arohan_ · x · 2026-11-13
Replying to arohan, Yaroslav Bulatov explains why he thinks Muon wasn't tried earlier: it doesn't fit the traditional preconditioning mindset. Using statistics computed on the same batch to precondition the current gradient seems intuitively wrong, so researchers likely assumed those statistics would be bad. A useful lens on what makes Muon unconventional.
More from Research
- François Fleuret: why isn't EMA normalization more popular than batch norm? — francoisfleuret · 2026-11-13
- Miles v0.2 Debuts at SGLang Summit: RL Training Across NVIDIA, AMD, TPU, Trainium — ying11231 · 2026-11-13
- Open robot learning book adds planning chapter covering CEM and loop control — GlenBerseth · 2026-11-13
- A 14-minute video traces one ChatGPT question from the Enter key to the GPU and back — Glass-Garden-5888 · 2026-11-13
- Reddit proposal: train a corrective LoRA for FLUX Klein from Krea-corrected outputs — Friendly-Fig-6015 · 2026-11-13
- Burkov's weekly: 94% of UK students use AI for assessed work, CUDA Rust, e-means — burkov · 2026-11-13