Self-distillation pitfall: reward rises, but the model hallucinates nonexistent studies
_lewtun · x · 2026-11-03
Lewis Tunstall notes the obvious alternative without a stronger teacher is self-distillation: run RLVR on the base model, keep correct rollouts, and SFT on them. Reward goes up, but final-answer-only correctness breeds hallucinations like citing studies and databases that don't exist — so the data itself must be inspected carefully.
Related event: HuggingFace Open-Sources FineDNA Recipe, Bypassing Teacher Distillation(4 posts)→
More from Research
- OpenEaNasir: AI-generated public-domain translation of the 4,000-year-old Ea-nāṣir complaint tablet — firasd · 2026-11-03
- GPT Astra-assisted proof improves Erdős Problem #357 bound, Lean-verified but not refereed — kevrussell · 2026-11-03
- RVCBench from UBC & Vector Institute stress-tests 18 voice cloning models across 18 robustness evaluations — jiqizhixin · 2026-11-03
- Counting mRNAs in living bacteria: the 2005 Golding paper that overturned the Poisson model of transcription — NikoMcCarty · 2026-11-03
- DeepSeek V4.1 Flash 1017 doubles ARC-AGI-3 score to 5.81%, leads open models — teortaxesTex · 2026-11-03
- Critics tear apart "Weight-Space Programming": Atari test spoon-feeds answers, Qwen case fails on Gemma — anshulkundaje · 2026-11-03