CMU Paper: AdamW Beats Muon in Finetuning When Pretraining Data Is Sufficient

A CMU paper (arXiv:2611.03633) finds that with sufficient pretraining data, AdamW outperforms Muon after finetuning, questioning whether Muon's pretraining advantages carry over to the finetuning stage.

2026-11-04 ~ 2026-11-04 · 2 related posts

• CMU Paper: With Enough Pretraining Data, AdamW Beats Muon After Fine-Tuning — AdtRaghunathan · 2026-11-04
• CMU study: at large pretraining data scales, AdamW beats Muon after fine-tuning — AdtRaghunathan · 2026-11-04