Developer reverse-engineers Dimuon optimizer, open-sources PyTorch implementation

Developer anpaure reverse-engineered the Muon fine-tuning optimizer variant (Dimuon / Polar Express) released days earlier by Thinking Machines' Tinker platform, recovering its internal mechanics and open-sourcing a PyTorch implementation (github.com/anpaure/dimuon). The replication covers 8 model families, rank 1–128, with errors below 1%.

Confirmed

The work stems from an idea proposed by @CreativeMath, with the reverse engineering implemented by anpaure.
Method: run Adam and Dimuon side by side on the same weights and batch, setting Adam's betas to 0 and lr = ε = 10,000, so its update ≈ −G; inverting the Adam formula then recovers the gradient G exactly—effectively using Adam as a gradient probe.
The recovered update direction is D = L · PE(L · M · C) · C, where L, M, C are the corresponding matrices (the post doesn't give full definitions).
Update magnitude follows lr·√r·(r/32)^0.175, matching measurements to roughly one part in a million.
The singular values carry Polar Express's fingerprint: an accompanying figure compares Muon vs. five steps of Polar Express on gpt-oss-20b, rank 16, first step (no history), showing clearly distinct curves.
Open-source repo: github.com/anpaure/dimuon; validation covers 8 model families, rank 1–128, with errors below 1%.

Why it matters

It shows that an optimizer released as closed source can be systematically reverse-engineered and open-sourced shortly after release, directly benefiting community understanding and reuse of cutting-edge fine-tuning techniques.
The singular-value fingerprint and the precise update-magnitude formula provide testable criteria for distinguishing Muon from Polar Express–style variants, aiding future optimizer research and improvement.

2026-10-29 ~ 2026-10-29 · 5 related posts

Primary sources

Dimuon optimizer reverse engineered and open-sourced, verified under 1% error across 8 model families — anpaure · 2026-10-29 03:09 (UTC+08:00)
Researchers reverse-engineer Tinker's finetuned Muon optimizer with under 1% error across 8 model families — anpaure · 2026-10-29 03:14 (UTC+08:00)

Trick: Adam with betas=0 and lr=ε=10,000 acts as an exact gradient observer — anpaure · 2026-10-29
Singular value fingerprints distinguish Polar Express from Muon's Newton-Schulz steps — anpaure · 2026-10-29
Reverse-engineering Dimuon: recovering Polar Express's exact update rule from weight deltas — anpaure · 2026-10-29
[source] Dimuon optimizer reverse engineered and open-sourced, verified under 1% error across 8 model families — anpaure · 2026-10-29
[source] Researchers reverse-engineer Tinker's finetuned Muon optimizer with under 1% error across 8 model families — anpaure · 2026-10-29