Direct-OPD Transfers Weak Model RL Gains to Strong Models
tokenbender · x · 2026-08-21
The paper proposes Direct On-Policy Distillation (Direct-OPD) to avoid expensive RL runs on large models. It runs RL on a weaker model and transfers the policy shift as an implicit reward to the stronger student. Empirically, Direct-OPD consistently improves stronger target models, boosting Qwen3-1.7B accuracy from 48.3% to 58.3%.
More from Models
- Claude Opus 5.5 effort levels tested: low reasoning iterates 9x faster than max — tom_doerr · 2026-10-16
- Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
- OpenAI ships GPT-Live-1.1: new voices in six languages, fewer confirmations — OpenAI · 2026-10-16
- Muse Spark model appears on LMArena, testable in Battle and Agent modes — arena · 2026-10-16
- Running a multi-model UI design arena: Paseo orchestrates four design drafts per model — vista8 · 2026-10-16
- Ilya Sutskever questions Meta license clause taxing MaaS providers above $20M revenue — ilyasu · 2026-10-16