How rollout staleness hurts on-policy distillation, and how to build a faster pipeline

burkov · x · 2026-11-17

Andriy Burkov highlights a systematic study of staleness in on-policy distillation pipelines. Post-training often spends more time generating long reasoning responses than updating weights; running generation, teacher scoring, and training in parallel improves hardware utilization but means the student trains on data from an older version of itself.

Key points:

Original post →

More from Infra

Infra channel →