Singular value fingerprints distinguish Polar Express from Muon's Newton-Schulz steps
anpaure · x · 2026-10-29
Follow-up in anpaure's optimizer forensics thread: update size follows lr·√r·(r/32)^0.175 to about 1 ppm, and singular values carry Polar Express's fingerprint. On gpt-oss-20b (rank 16, first step, no history), five Polar Express steps produce a markedly different update distribution along G's singular vectors than Muon's five Newton-Schulz steps, with Tinker Dimuon's measured points landing on the Polar Express curve. Surprise finding: even with β = 0 a history effect persists — the previous momentum also supplies a preconditioner.
More from Research
- Google DeepMind engineer shows five CUDA optimizations that speed up 3D Gaussian Splatting by 10.5x — janusch_patas · 2026-10-29
- AI Audit Finds 2,035 Errors in 138 Papers by Economists Saez and Zucman — sebkrier · 2026-10-29
- Granary-v2 open speech model expands to 48 languages with open data and pipeline — rdesh26 · 2026-10-29
- Tinker launches Dimuon, a Muon-based optimizer that beats AdamW on LoRA fine-tuning — xennygrimmato_ · 2026-10-29
- Six CUDA kernel optimizations squeeze another 1.78× speedup out of 3DGS training — gkopanas · 2026-10-29
- Cross-lingual LLM effects paper wins Outstanding Paper award at EMNLP 2026 — anas_ant · 2026-10-29