Hugging Face trains a DNA understanding model on Qwen3-14B with 300B hybrid tokens
josh_wills · x · 2026-11-04
Hugging Face built FineDNA, a DNA understanding model: 300B tokens of hybrid DNA + natural language midtraining on Qwen3-14B yields a strong base for domain-specific post-training. A key challenge was the lack of a stronger model to distill from; the team shares what worked when building the SFT and RL subsets.
More from Research
- Vulnerability research is a topology problem: reusing local proofs across audits — dyn___ · 2026-11-04
- Agents will soon edit their own weights and memory — but who studies when to use which? — sarahcat21 · 2026-11-04
- Hand-writing all weights of a 100k-param model beats training it at spotting fake legal citations — AaronBergman18 · 2026-11-04
- Stanford reran 50 famous LLM findings on 20 models — 44% no longer hold — heyshrutimishra · 2026-11-04
- Gradient-Sealed Unlearning aims to stop fine-tuning from restoring harmful model capabilities — burkov · 2026-11-04
- New method forecasts GSM8K accuracy within 3 points using 5600x less compute — kenziyuliu · 2026-11-04