Tsinghua's One-Shot OPD Matches Full Distillation with 16 Queries

Tsinghua researchers propose One-Shot OPD, showing that on-policy distillation with as few as 16 queries recovers most of the benefits of full-data distillation, with single-query training still improving over hundreds of steps.

2026-09-04 ~ 2026-09-04 · 2 related posts

• One-Shot OPD: a single query keeps on-policy distillation improving for hundreds of steps — ParshinShojaee · 2026-09-04
• Tsinghua team: 16 diverse queries match full-data on-policy distillation — 青稞AI · 2026-09-04