Solving Privilege Illusion in Policy Distillation: DAPD Boosts Model Performance
Shanghai-AI-Laboratory · hf · 2026-08-04
On-policy self-distillation (OPSD) is widely used for language-model post-training. However, it can induce a 'privilege illusion': the student learns privilege-dependent behavior from the teacher during training, but behaves as if the privileged information is still available during inference, ultimately degrading performance.
Researchers from Shanghai AI Lab identify information asymmetry between the privileged teacher and the student at inference as the root cause. They propose DAPD (Dual-Anchored Policy Distillation):
- Dual-Path Anchoring (DPA): Introduces a self-conditioned bridge and aligns reference and rollout behavior along two matched-information paths, preventing privilege-dependent behavior transfer.
- Dual-Source Anchoring (DSA): Applies these paths bidirectionally, reducing reliance on privileged reference guidance while preserving correctness supervision.
Experiments show DAPD outperforms OPSD by +2.00 points on average across tasks, with gains persisting across scales from 4B to 32B models.
More from Research
- Weekend Project: RL-Trained 4B LLM Rewrites AI Text to Fool Open-Source Detectors — matthen2 · 2026-08-04
- NUS Creates Octopus-Inspired Swimming Robot Driven by Just Two Motors — lukas_m_ziegler · 2026-08-04
- Native Tool Calling and Correct Sampling Params Boost LLM Evals by >30 Points — xeophon · 2026-08-04
- KlingTeam Proposes Cross-Category Video Motion Transfer Framework with New Dataset — KlingTeam · 2026-08-04
- DeepVoyager-VL: Building Long-Horizon Multimodal Agents Without RL — Huanyao Zhang · 2026-08-04
- Peking University Introduces ReBA for Vision-Language MoE Load Balancing — PekingUniversity · 2026-08-04