Skipping the teacher: code-grounded reasoning traces beat self-distillation (67% vs <60%)
_lewtun · x · 2026-11-03
Lewis Tunstall shares lessons from building the FineDNA dataset: instead of distilling from a teacher model, they assembled each training explanation from facts computed in code plus the model's own sense of how unusual a mutation looks. With these grounded reasoning traces, accuracy reached 67%, while the self-taught version stalled below 60%.
Related event: HuggingFace Open-Sources FineDNA Recipe, Bypassing Teacher Distillation(4 posts)→
More from Research
- OpenEaNasir: AI-generated public-domain translation of the 4,000-year-old Ea-nāṣir complaint tablet — firasd · 2026-11-03
- GPT Astra-assisted proof improves Erdős Problem #357 bound, Lean-verified but not refereed — kevrussell · 2026-11-03
- RVCBench from UBC & Vector Institute stress-tests 18 voice cloning models across 18 robustness evaluations — jiqizhixin · 2026-11-03
- Counting mRNAs in living bacteria: the 2005 Golding paper that overturned the Poisson model of transcription — NikoMcCarty · 2026-11-03
- DeepSeek V4.1 Flash 1017 doubles ARC-AGI-3 score to 5.81%, leads open models — teortaxesTex · 2026-11-03
- Critics tear apart "Weight-Space Programming": Atari test spoon-feeds answers, Qwen case fails on Gemma — anshulkundaje · 2026-11-03