EOS mismatch drives length inflation in on-policy distillation, HF study shows fix
Yuxiao Yang · hf · 2026-09-18
A new study dissects "length inflation" in on-policy distillation, where student responses grow excessively long or exhaust the generation budget.
Key findings:
- A major source is termination-token mismatch: across Qwen3, Llama, and Gemma, student and teacher place stopping probability on different EOS tokens even when declared stopping sets match, suppressing the student's preferred termination.
- Aligning decoding stopping sets alone is insufficient; treating functionally equivalent EOS tokens as a shared semantic stopping action substantially mitigates the issue across all three families.
- Stage-wise OPD analysis on K2-Horizon shows termination preferences shift during training, plus a late-stage length inflation that persists beyond termination alignment.
An implementation with the proposed termination-handling corrections is released.
Related event: EOS Token Mismatch Causes Length Inflation in Online Distillation(3 posts)→
More from Research
- Fields Medalist Villani on OpenAI's Millennium Problem: 'A Cataclysm Like Math Has Never Known' — GregCook2011 · 2026-09-20
- lateinteraction: Papers are just timestamped formats — the real projects live elsewhere — lateinteraction · 2026-09-20
- Two high school students, aided by AI, report progress on a problem studied by Fields medalist June Huh — IgorCarron · 2026-09-20
- GPU Programming Diary: Revisiting the Classic CUDA Matmul Optimization Worklog and MIT's Sparsity Lecture — NandoDF · 2026-09-20
- New paper finds 'anti-grokking': test accuracy collapses back to chance after successful generalization — sytelus · 2026-09-20
- With 50k+ ICLR submissions, researchers propose hybrid human-LLM peer review orchestration — HamedSHassani · 2026-09-20