Optimizer Geometry Changes Scaling Laws: Muon Outperforms AdamW in Hard-Rank Growth
YouJiacheng · x · 2026-08-18
NYU research reveals that realized representation capacity in Transformers emerges from architecture–optimizer interaction, not just architecture alone. Different optimizers (AdamW vs. Muon) yield distinct spectral scaling laws under fixed conditions. Hard-rank scaling sharply differentiates optimizers: AdamW shows weak scaling, while Muon achieves much stronger hard-rank growth, converting added FFN width into dominant usable capacity more efficiently.
More from Research
- AURORA-LM: Diffusion Models Master High-Fidelity Text Representations — jiqizhixin · 2026-08-18
- Training model without data inspection? The model trains you — kalomaze · 2026-08-18
- Study Finds 3.8M Agent Skill Files Across 282K GitHub Repos — dair_ai · 2026-08-18
- 300-Page Monograph: Engineering Reliable Coding Agents as Systems — heyneighbor · 2026-08-18
- Tsinghua and ByteDance's CUDA Agent Writes Better CUDA Than Human Experts — anselm · 2026-08-18
- Paper on "Machine Studying" Explores New Paradigm for AI Agents — lateinteraction · 2026-08-18