Echelon optimizer: Muon-level results with up to 10.5× faster steps, Lean-proven

xennygrimmato_ · x · 2026-11-17

Evolving Machines introduced Echelon, a rank-anchored reformulation of Muon that orthogonalizes only the top r subspace and rescales the tail, cutting cost from cubic in layer width to O(mnr).

Key numbers: 10.5× faster optimizer steps at 70B-class shapes (33.9s → 3.23s on one RTX PRO 6000); 2.5× faster steps and 14% shorter training steps in a real 1.3B run with validation loss (4.375) within seed noise of Muon across 3 seeds; 35% less time per training step on DeepSeek V4.1 Flash on 16 B300s (31% on 24), excluding communication.

The team paired the algorithm with the first Lean-checked convergence proof for spectral steepest descent.

Related event: Echelon Optimizer Speeds Up Muon Up to 10.5x(2 posts)→

Original post →

More from Infra

Infra channel →