Genomics AI researcher: distillation gains are marginal, better losses and data sampling still ahead
anshulkundaje · x · 2026-09-19
Stanford researcher Anshul Kundaje responds to questions about AlphaGenome vs. pre-distilled genomics models: the pre-distilled models are publicly shared and extensively compared. Variant correlations (signed Pearson's r) look basically identical across models, suggesting distillation mainly reshuffles noise. He says substantial gains remain in fine-tuning via better losses and refined data sampling, with more results coming soon.
Related event: Researchers Defend Distilled Genomic Models, See Room for Fine-tuning(2 posts)→
More from Research
- Sarah Hooker shares a Colab notebook that 'invents' a dataset in a few lines of code — sarahookr · 2026-09-20
- A 0.62-AUC classifier helped discover 6 new altermagnets: AI's job is making one loop step cheap — bravo_abad · 2026-09-20
- "Self-Evolving Search Index" paper sparks SEO buzz: indexes that auto-adapt to abuse — gaganghotra_ · 2026-09-20
- Researcher's New Routine: Most Time Now Spent Reading Papers Written by His Own Models — generativist · 2026-09-20
- FrontierSWE v2 opens 24.1-point gap: Claude Fable 5.1 scores 56.29% vs GPT-5.6's 32.2% — geoffwolfe · 2026-09-20
- 22M local model beats JEV 93% vs 80% on Banking77 in 8ms on CPU — Prompt Engineering · 2026-09-20