OPMD distillation beats NTP mid-training, lifting Qwen on MATH-500 and Minerva
kalyan_kpl · x · 2026-11-11
A new arXiv paper proposes On-Policy Mid-Training Distillation (OPMD), reframing mid-training in continual pre-training from offline text imitation into active, query-driven knowledge acquisition.
- Motivation: In a controlled study on the same 1B-token math corpus, standard next-token prediction (NTP) shows model-dependent effects — it improves Llama-3.2-3B's math benchmarks but substantially degrades Qwen3-1.7B-Base, meaning a fixed recipe can backfire on different models.
- Method: At selected anchor positions, OPMD samples trajectories from the evolving student policy and applies dense token-level reverse-KL rewards from a frozen teacher, training on states the model actually visits rather than only ground-truth prefixes, reducing the train-generation mismatch of teacher-forced NTP.
- Results: On MATH-500 and Minerva, OPMD lifts the Qwen student above its base performance and beats the NTP baseline, which degrades on both benchmarks.
- Practicality: OPMD plugs into existing CPT pipelines with no added inference-time overhead, offering a model-aware alternative to passive domain adaptation.
More from Research
- Scandinavian Journal of Statistics editors reflect on publishing in the age of generative AI — michaelchchoi · 2026-11-11
- Claude Opus 5.5 runs an architecture search and finds AttnRes-Canon, a new top Transformer design — SonglinYang4 · 2026-11-11
- 5 research habits worth knowing: read papers backwards, reproduce first, wire coding agents via MCP — cneuralnetwork · 2026-11-11
- Running Qwen2.5-Coder-3B on Android: Jupyter + Pydantic Evals on-device eval pipeline — alex_b3b · 2026-11-11
- cloneofsimo releases a video explaining determinantal point processes — cloneofsimo · 2026-11-11
- Mapping Qwen3-8B's internal touch body map into an anatomical plate — repligate · 2026-11-11