New Self-Distillation Method Boosts LLM Self-Correction Without Supervision
burny_tech · x · 2026-08-08
Traditional self-distillation methods for LLMs often rely on gold answers, verifier rewards, or stronger teacher models. A new paper, "On-Policy Self-Distillation without Any Supervision," introduces a fully self-supervised approach.
The core mechanism involves:
- Sampling multiple solutions per problem
- Majority-voting to generate a pseudo-answer
- Distilling the pseudo-solution-conditioned distribution into the rollouts that disagreed
This provides dense on-policy corrections without labels. Experiments show that on Qwen3 math tasks in non-thinking mode, this method outperforms supervised OPSD and GRPO by 3.2 and 8.9 points, respectively.
Related event: ByteDance Proposes On-Policy Self-Distillation for LLMs(2 posts)→
More from Research
- ALPINE: a 35K-parameter few-shot architecture that beats MAML with 27-53% fewer parameters — Neeraj Yadav · 2026-09-23
- Fudan's ScriptMoE recognizes text in 229 languages, beating VLMs with far fewer parameters — Fudan-University · 2026-09-23
- slime merges Score Centering, a first-principles fix for RL instability when train and sampling policies diverge — hsu_byron · 2026-09-23
- Google's Light Heads Cuts YouTube Recommender Experiment Cycles from Weeks to Days — _reachsumit · 2026-09-23
- Orthrus Serves Embedding and Generation in One GPU Batch, 4.52x RAG Throughput — _reachsumit · 2026-09-23
- Spotify: Behavioral Stats Boost LLM Reranking 13.3% but Teach It Shortcuts — _reachsumit · 2026-09-23