Where-OPD paper: distillation teacher privileged by knowing where to look, not a better view
abursuc · x · 2026-10-06
A new paper, Where-OPD, rethinks on-policy self-distillation. In these methods the student learns from a privileged version of itself, and recent approaches grant that privilege via a better view of the image. Where-OPD asks: what if, instead of a better view, the teacher knew where to look — making attention location itself the source of privileged information. Full details in the thread.
More from Research
- Survey: 65% of Japanese seniors prefer robot-assisted nursing homes, willing to pay 8% more — HealthcareLdr · 2026-10-06
- AI Scholar Yi Ma Proposes Rolling 10-Paper Cap on arXiv to Curb Paper Flooding — YiMaTweets · 2026-10-06
- FlashDexRetarget: one RL policy retargets hand demos at 90% success, ~100x less compute — KyleMorgenstein · 2026-10-06
- Stanford ACE team unveils Sentry: failure tips in context hurt LLM agents, +39% gains — StanfordAILab · 2026-10-06
- From Kaggle Champion to ULMFiT: How Jeremy Howard Rewrote Language Model Training — bigaiguy · 2026-10-06
- Training on a Post-Trained Model Often 'Fries' It, Causing Reality Drift — Sauers_ · 2026-10-06