Analysts spot mid-training optimizer switches from Adam to Muon

Analysts spotted frontier training runs apparently switching from Adam to Muon mid-training—contradicting Kimi's Moonlight paper—and found Kimi's minor k3 update involved a full re-pretrain with identical architecture, suggesting a data refresh.

2026-09-22 ~ 2026-09-22 · 3 related posts

• Kimi's minor bump was a full pretrain: two-phase training splits omni data from the pro model — stochasticchasm · 2026-09-22
• Analysis suggests mid-training Adam-to-Muon switch, contradicting Moonlight paper findings — stochasticchasm · 2026-09-22
• Adam-to-Muon mid-training switch sparks debate, seemingly contradicting Moonlight paper — stochasticchasm · 2026-09-22