Muon-RT reuses the EMA trajectory: +2.6 points, 42% less memory

MUON-RT Reusing the Teacher's Trajectory for Momentum-Free Muon for LLM Self-Distillation

Aryan Das, Vinay Kumar Verma, Aakansha Mishra

cs.LG, cs.AI, cs.CL, stat.ML

2026-10-30

Muon-RT replaces Muon's momentum buffer with the EMA teacher-student displacement. On Qwen2.5-7B SDFT it gains 2.6 points over AdamW and cuts persistent memory by 42%.

What problem this solves

SDFT teaches a new skill by giving the teacher an expert demonstration as extra context. SDPO does the on-policy version, using feedback from a successful sample. In both, the teacher is an exponential moving average (EMA) of the student weights: after every step, θT ← τθT+(1−τ)θS. With τ near 1 the teacher inches forward.

The student samples the completion. The teacher scores the same tokens with that extra context. The loss is the gap between the two next-token distributions.

The teacher is already a stored parameter trajectory. Muon still keeps a momentum buffer the size of every hidden matrix. Muon updates only the 2D weights inside Transformer blocks. It folds the gradient into the buffer, then runs five Newton-Schulz iterations to flatten the nonzero singular values and produce an approximately orthogonal step. Embeddings, the output head, norms, and biases stay on AdamW. Each hidden matrix therefore holds three persistent copies: student, teacher, and momentum. AdamW holds four, once its two moment buffers are counted. On Qwen2.5-7B-Instruct that is 51.6 GB per GPU for momentum Muon and 64.5 GB for AdamW.

Momentum smooths gradients. The teacher smooths weights. The stand-in for the buffer is the displacement WT−WS.

Method

Unrolling the EMA turns that displacement into a discounted sum of updates that were actually written into the student. Recent steps count more. Muon momentum stores raw gradients. The displacement stores directions that have already been orthogonalized and applied. Dropping it into the momentum slot feeds the optimizer its own output. Adding it to the gradient before Newton-Schulz is harder to control: the iteration lifts weak singular directions up to the strong ones, and α no longer sets the mix.

Muon-RT keeps the two signals apart and adds them after orthogonalization.

The extra matrices are temporary and freed after each block, so the persistent state loses the momentum buffer. One optimizer step takes 2.3 seconds, about 2% of a training step dominated by generation.

If update noise is uncorrelated in time, the discounted sum cuts its variance to at most (1−τ)/(1+τ) of a single step. The best-aligned direction for the current gradient is still that gradient's polar factor. The teacher adds smoothing across steps. Under smoothness, a PL inequality, bounded objective drift, and alignment between the update and the gradient, the expected objective gap contracts linearly toward a residual neighborhood. Those constants are not estimated on a language-model loss.

Results

SDFT runs on Qwen2.5-7B-Instruct with two H100s. Tool use is 252 steps, science questions 166 steps, 32 prompts per step, two epochs. The learning rate is the better of 3e-5 and 5e-5, α is 0.1, 1−τ is 0.01, and each setting uses three seeds. The AdamW reference is taken from the original SDFT paper and, under this shared 32-bit teacher, sits inside run-to-run variation.

Muon-RT has the highest new-task scores: 74.2% on tool use and 71.8% on science. SDFT with AdamW is at 70.6% and 70.2%, a 2.6 point average gain. Momentum-free Muon is at 69.1% and 69.0%, so the teacher branch accounts for 5.1 and 2.8 points. Momentum at μ=0.95 is lower, 66.9% and 68.3%, 5.4 points behind on average. A large part of that 5.4 is momentum hurting: turning it on drops Muon by 0.7 to 6.2 points on the two tasks. Against momentum-free MuonClip, which uses the same memory, the margin is only 1.0 and 0.8 points.

TaskOptimizerNew taskPrior avg
Tool useSDFT+AdamW70.665.4
Tool useMuon μ=069.164.3
Tool useMuonClip μ=073.263.5
Tool useMuon-RT74.263.0
ScienceSDFT+AdamW70.264.5
ScienceMuon μ=069.064.8
ScienceMuonClip μ=071.065.4
ScienceMuon-RT71.865.3

The prior average covers HellaSwag, HumanEval, IFEval, MMLU, TruthfulQA, and WinoGrande. Self-distillation runs stay within 2.4 points of each other and above SFT: 65.3 vs 53.4 on science, 63.0 vs 56.3 on tool use. On science, Muon-RT's HumanEval is 70.7 and its six-task average of 65.3 beats AdamW's 64.5. On tool use the new task gains 3.6 points while the six-task average loses 2.4. IFEval falls from 71.9 to 63.7. The prior average that remains is about 96% of AdamW's.

On tool use, Muon-RT reaches 74.2% at step 100 of 252. MuonClip needs step 250 to reach 73.2%. Momentum-free Muon never passes 69.1%.

SDPO moves to Olmo-3-7B-Instruct on the biology and materials slices of SciKnowEval. The score is the best avg@16 within 1 hour and within 5 hours, meaning mean accuracy over 16 samples. Across the four cells Muon-RT is 2.0 points above SDPO with AdamW. The 1-hour margin is 0.7 to 1.1 points. At 5 hours it is 2.4 and 3.7. Materials at 5 hours is 82.8%, above every listed baseline, including AdamW at 79.1%. Biology at 5 hours is 55.2%, above SDPO+AdamW at 52.8% and below GRPO at 55.8%. GRPO trains on a different signal.

Persistent memory per GPU, measured between steps after gradients are freed: AdamW 64.5 GB, momentum Muon 51.6 GB, Muon-RT 37.7 GB. That is 26.8 GB (42%) under AdamW and 13.9 GB (27%) under momentum Muon. Momentum-free Muon and MuonClip also use 37.7 GB and score 0.8 to 5.1 points lower. Projected to 10T parameters with every state in 32 bits and gradients included, AdamW, momentum Muon, and Muon-RT are 200 TB, 160 TB, and 121 TB, so Muon-RT is 40% and 25% smaller than the first two. The 7B measurement excludes gradients, so the two percentages do not share an accounting.

α peaks sharply at 0.1. On tool use, α of 0, 0.05, 0.1, 0.2, and 0.4 scores 69.1, 67.0, 74.2, 68.0, and 67.0. α=0.05 is worse than using no teacher. Science scores 69.0, 66.5, 71.8, 66.8, and 65.4. Mixing before Newton-Schulz, keeping the radial component, or gating α by cosine drops tool use to 72.5, 70.5, and 69.2. QK-Clip lifts momentum-free Muon from 69.1 to 73.2 on tool use and drops Muon-RT from 74.2 to 66.0. Clipping rescales the student query only, so the displacement picks up a jump that is not a learning step.

Why it matters

When a run already keeps a parameter-aligned EMA, Muon-RT is an optimizer swap. The SDFT and SDPO losses stay put. The state that goes away is the hidden-matrix momentum. At 7B it uses the memory of momentum-free Muon and beats Muon that actually stores a momentum buffer.

Without that aligned teacher the swap does not apply. Keeping a teacher only to delete momentum does not reduce memory. The 42% figure assumes a 32-bit teacher. The original SDFT setup stores the teacher in bfloat16, and the same percentage does not carry over.

Limitations

The runs cover two 7B families and four tasks. Linear contraction uses smoothness, PL, and alignment assumptions, and the constants are not fit on a language-model loss. The check that the teacher direction aligns better and has a smaller second moment uses the key projection from layer 10 plus synthetic noise with a known mean, not real distillation gradients.

The 5.4 point comparison is against μ=0.95. Momentum-free MuonClip, the stronger baseline at the same memory, is only about a point behind, and tool-use IFEval falls from 71.9 to 63.7. α=0.05 is worse than α=0, so the optimum is sharp, and SDPO reuses 0.1 with no third task to see whether the peak moves. Three seeds were run. The tables do not report the spread.

At τ=0.99 a teacher step is about 1e-5 relative to the weights. bfloat16 only resolves relative changes down to about 4e-3, so most EMA steps round away. On one key projection the distances from initialization are 2.15 for the student, 1.37 for a 32-bit teacher, and 0.66 for a bfloat16 teacher, and 79% of the bf16 entries never change. The 32-bit teacher adds 7.6 GB per GPU. Against the original two-bfloat16 SDFT layout, the memory saved is smaller than 42%.

Terms

Source

Related papers

All paper explainers