Muon optimizer may nearly halve LLM training compute, challenging a decade of AdamW
bycloud · youtube · 2026-10-24
bycloud's video argues Muon is one of the most underrated LLM breakthroughs of the past two years: after a decade of AdamW dominance, Muon could cut training compute nearly in half and make previously overlooked architectures viable.
Key sources covered:
- Keller Jordan's original Muon blog and the paper "Muon is Scalable for LLM Training"
- Moonshot's Moonlight, a practical validation of Muon's scalability
- DeepSeek-V4 material and studies showing Muon learns more robust, transferable features than Adam
More from Infra
- ik_llama.cpp Fork Update: Matches Strata Speeds on a 6GB RTX 2060 — IceFog72 · 2026-10-25
- Watch live: YC Paper Club on the LLM inference stack — ycombinator · 2026-10-25
- YC Paper Club on inference: every training FLOP costs ~6x per year in serving, a $220B stack — ycombinator · 2026-10-25
- Running DeepSeek at 10 tok/s on an RTX 5090 costs more in electricity than API calls — DeMischi · 2026-10-25
- Next frontier: un-nerfed instruction sets would make AMD 'unstoppable' — TheZachMueller · 2026-10-25
- NVIDIA un-nerfs consumer cards: 'you bought the card and get the whole card' — TheZachMueller · 2026-10-25