Meta paper reveals Muon optimizer scaling properties for large DiTs
iScienceLuvr · x · 2026-08-24
A new paper by Meta and USC examines Muon's scaling behavior on DiTs from 1.3B to 15B parameters. While Muon retains optimization and quality advantages over AdamW at scale, the computational overhead of full Newton-Schulz iterations is substantial. They propose Periodic Row-wise Muon, performing full spectral updates periodically and low-cost row-wise constrained updates in between to mitigate overhead.
More from Research
- Open source Pelican SVG Env quantifies model drawing abilities — SergioPaniego · 2026-08-24
- 17-year-old who independently pretrained models gets ICML paper accepted — HanchungLee · 2026-08-24
- CAS-Spawned ScienceClaw Launches AutoProject: AI Research Agents Move From Tasks to Whole Projects — 量子位 · 2026-08-24
- British Library Releases Million-Scale Book Image Dataset on Hugging Face — biglam · 2026-08-24
- Turkish Court Decisions: 10M+ Legal Texts Dataset Trends on Hugging Face — hamzabagirsakci · 2026-08-24
- UniSpace: Unified Visual Representation without VAE — meituan-longcat · 2026-08-24