HuggingFace Open-Sources FineDNA Recipe, Bypassing Teacher Distillation
HuggingFace's Lewis Tunstall detailed how FineDNA avoids self-distillation hallucinations by building reasoning chains from code-computed facts, outperforming self-distillation, and fully open-sourced its 300B-token mid- and post-training recipe.
2026-11-03 ~ 2026-11-03 · 4 related posts
- Episode 1: Hugging Face Open-Sources DNA Model Carbon-X as Stanford Professor Questions Its Benchmarks(2026-10-22, 33 posts)
- Episode 2: Hugging Face Open-Sources Carbon-X, a 14.8B DNA Language Model(2026-10-25, 3 posts)
- Episode 3: Hugging Face Open-Sources FineDNA Dataset and Carbon Models; Carbon-X Beats GPT-6.1 on DNA Benchmarks(2026-11-01, 11 posts)
- Episode 4: HuggingFace Open-Sources FineDNA Recipe, Bypassing Teacher Distillation(2026-11-03, 4 posts)
- Self-distillation pitfall: reward rises, but the model hallucinates nonexistent studies — _lewtun · 2026-11-03
- Skipping the teacher: code-grounded reasoning traces beat self-distillation (67% vs <60%) — _lewtun · 2026-11-03
- FineDNA post-training: grounded reasoning traces beat teacher distillation, 67% vs <60% — _lewtun · 2026-11-03
- HuggingFace opens its FineDNA recipe: mid-training on 300B tokens breaks the distillation ceiling — _lewtun · 2026-11-03