Muon optimizer becomes the default as new models drop Adam entirely
stochasticchasm · x · 2026-09-11
stochasticchasm notes the "head-wise Muon shift" is becoming the default, observing that the newest model uses no Adam at all—plus a notably large engram table. Another data point in Muon-style second-order optimizers displacing AdamW in frontier training runs.
Related event: Leaked DeepSeek V4.1 Flash Tech Report Reveals KV Cache-Obsessed Flagship(44 posts)→
More from Research
- Stanford's PaperFormalizer turns research papers into Lean-verified versions with author in the loop — StanfordUniversity · 2026-10-27
- JockeyBench uses Japanese horse racing to test LLM agents — all betting strategies lose money — Zhongwei Xie · 2026-10-27
- Cohere's Invent-A-Dataset paper: zero-seed synthetic data beats frontier APIs on quality and diversity — idanbeck · 2026-10-27
- Hugging Face TRL makes score centering + masked importance sampling default in async GRPO — _lewtun · 2026-10-27
- AutoResearch agent's weird training curve gets dug into instead of discarded — ZimingLiu11 · 2026-10-27
- EMNLP 2026: sub-billion-parameter LMs show superhuman entity tracking, human-like fleeting memory — GretaTuckute · 2026-10-27