Meta paper reveals Muon optimizer scaling properties for large DiTs
iScienceLuvr · x · 2026-08-24
A new paper by Meta and USC examines Muon's scaling behavior on DiTs from 1.3B to 15B parameters. While Muon retains optimization and quality advantages over AdamW at scale, the computational overhead of full Newton-Schulz iterations is substantial. They propose Periodic Row-wise Muon, performing full spectral updates periodically and low-cost row-wise constrained updates in between to mitigate overhead.
More from Research
- Halcyon: no text pretraining, beats LlamaGen-XL/8 with 18× less compute — JosephJacks_ · 2026-10-16
- GPT-6.1 Sol audit finds 6% of arXiv papers had false main theorems, down to 1.5% recently — nitarshan · 2026-10-16
- One tiny word changes the conversation: how LLMs pick up on 'just' — StellaLisy · 2026-10-16
- SEAR: first large-scale controlled study of self-evolving robot agents, 7 LLMs, 330 tasks — xiye_nlp · 2026-10-16
- After Claude-assisted elliptic curve records fell, EC Rank challenge opens 156 record slots — Jsevillamol · 2026-10-16
- Oxford neuroscientist swaps fetal MRI for ultrasound + ML models — AnnaCiaunica · 2026-10-16