Muse Training Recipe: Full Logit Distillation Details Revealed
Details reveal Muse models were pretrained with logit distillation from Muse Spark using top-256 logits for the first 95% of training, switching to ground-truth tokens only in the final 5%.
2026-10-20 ~ 2026-10-21 · 2 related posts
- Episode 1: Muse Training Recipe: Full Logit Distillation Details Revealed(2026-10-20, 2 posts)
- Episode 2: Muse Glimmer's Mid-Training and SFT Recipe Revealed: 131k Context, Mixed Data(2026-10-21, 2 posts)
- Muse Spark training detail: pure KL logit distillation for 95% of pre-training — andrew_n_carr · 2026-10-20
- Pretraining recipe: 525 TPP top-256 logit codistillation from Spark, final 5% on ground-truth tokens — TimDarcet · 2026-10-21