Router optimizer split: DeepSeek keeps Muon while Qwen switched to AdamW after instability
Nouamanetazi · x · 2026-11-06
The MoE handbook compares router optimizer choices: DeepSeek-V4 updates the router with Muon like the rest of the model, while Qwen3.8-Next moved it to AdamW after Muon destabilized the router early in training. With per-expert Muon, a rarely picked expert moves about as far as the busiest one (0.61 under AdamW at 16 tokens).
Related event: Hugging Face Releases Free MoE Training Handbook Built on 800K GPU Hours(20 posts)→
More from Infra
- Tech rout: TL20 down 19% in a week; author says NVDA fell 60% before rising 1,062% — TiernanRayTech · 2026-11-06
- Qwen3.8-27B hits 381 tok/s on a single RTX 3090 via inference software alone — Redo · 2026-11-06
- Dolphin Networks details 3.8× throughput gains running DeepSeek-V4.1 Flash on PCIe GPUs — AlpinDale · 2026-11-06
- Lawyer runs DeepSeek-V4.1-Flash on 8x 2018 V100s: 215 tok/s on code, 524K context fully in VRAM — TumbleweedNew6515 · 2026-11-06
- Hyperkernel generates SOTA megakernels across 11 domains, now on GitHub — MillionInt · 2026-11-06
- Musk: space will be the cheapest place to run superintelligence in 2-3 years — XFreeze · 2026-11-06