Norm-swap experiments trace Muon early-decay gains to weight norms

ziv_ravid · x · 2026-11-19

Direct mechanism test at 162M: replaying the early-decay step-size trajectory with fixed norms and no decay yields a similar gain, and swapping norms between runs moves the gain with them (96% one way, 97% the other) — confirming the weight-norm/effective-step pathway.

Related event: Temporary Early Weight Decay in Muon Pretraining Shows Delayed Payoff(8 posts)→

Original post →

More from Research

Research channel →