Exactly solvable theory shows when Muon beats Adam: steep spectra, hard targets
fpedregosa · x · 2026-11-04
A new ML theory work builds an exactly solvable theory of Muon vs. SignSGD, answering when Muon beats Adam.
- The answer depends on the data: Muon wins on steep spectra with hard targets; SignSGD wins on flat spectra or easy targets.
- The results have direct implications for Muon and Adam on anisotropic data (e.g., power-law spectra). Posey Paquet presented the work as a 10-tweet theory thread; collaborator Cy Paquette called out its "cool implications for Muon and Adam with anisotropic data."
More from Research
- 132-Page Tutorial on Generative Models for Music Audio Lands on Hugging Face — amaai-lab · 2026-11-04
- Open-source TaoMate-FVSR restores face videos with audio conditioning and visual dubbing — Apprehensive_Sky892 · 2026-11-04
- Stanford's ProteoCraft hits 82.9% experimental hit rate on PD-L1 protein design — Lianhuiq · 2026-11-04
- Chetty's Opp Insights team uses ML 'digital twins' to recover causal effects, validated by RCTs — paulnovosad · 2026-11-04
- D&A turns any VLM into a zero-shot robot reward model, hitting SOTA on 3 benchmarks — KyleMorgenstein · 2026-11-04
- Your job as a scientist is disproving your own results: baselines, leakage, reward hacking — tw_killian · 2026-11-04