Echelon optimizer: Muon-level results with up to 10.5× faster steps, Lean-proven
xennygrimmato_ · x · 2026-11-17
Evolving Machines introduced Echelon, a rank-anchored reformulation of Muon that orthogonalizes only the top r subspace and rescales the tail, cutting cost from cubic in layer width to O(mnr).
Key numbers: 10.5× faster optimizer steps at 70B-class shapes (33.9s → 3.23s on one RTX PRO 6000); 2.5× faster steps and 14% shorter training steps in a real 1.3B run with validation loss (4.375) within seed noise of Muon across 3 seeds; 35% less time per training step on DeepSeek V4.1 Flash on 16 B300s (31% on 24), excluding communication.
The team paired the algorithm with the first Lean-checked convergence proof for spectral steepest descent.
Related event: Echelon Optimizer Speeds Up Muon Up to 10.5x(2 posts)→
More from Infra
- YC F26 Startup Maritime Launches a Managed Cloud for Running AI Agents — ycombinator · 2026-11-17
- Why Big Tech Fears Local AI: Security, Performance and Cost Advantages — DavidLinthicum · 2026-11-17
- Microsoft Details Maia Dataflow Architecture: 4719 Tflop/s fp8, Over 99% of Peak — thoefler · 2026-11-17
- Open-Weight Models Now Handle 30% of Enterprise Usage, Up From Near Zero Since July — Terminator857 · 2026-11-17
- What developers should check before deploying models on Cerebras — SpottyMoggy · 2026-11-17
- SkyPilot makes The Information's top 50 startups list at $345M valuation — skypilot_org · 2026-11-17