Tinker launches Dimuon, a Muon-based optimizer that beats AdamW on LoRA fine-tuning
xennygrimmato_ · x · 2026-10-29
Thinking Machines' Tinker platform released Dimuon, a new optimizer built for LoRA fine-tuning, reporting that in SFT tests it often reaches lower loss than AdamW in fewer steps. It's live on Tinker with a feedback request and an optimizers guide.
Via API probing, one researcher recovered Dimuon's update rule: Dimuon = Muon + three changes — delayed diagonal conditioning (row/column scaling from the previous momentum), Polar Express replacing Newton–Schulz, and a fixed step size set by rank.
More from Research
- BlackboxNLP orals: knowledge attribution probing and CoT monitorability — iatitov · 2026-10-29
- Ivan Titov to speak at BlackboxNLP on what post-training for reasoning actually does to LLMs — iatitov · 2026-10-29
- New Analysis Explores Why Muon's Orthogonalized Updates May Fix Factual Recall Imbalance — burkov · 2026-10-29
- BlackboxNLP kicks off at EMNLP2026 Budapest, opening keynote on how LLMs read — gsarti_ · 2026-10-29
- QUAN paper: real-valued quaternion convolution nets SOTA in oriented detection — Bryce Grant · 2026-10-29
- Tim Vieira Explains Conditional Poisson Sampling with Verified Identities and Interactive Visuals — xtimv · 2026-10-29