Carbon-X DNA model slammed: worse than random on 8/10 clinical variant cases
anshulkundaje · x · 2026-10-23
Hugging Face's Loubna Ben Allal launched Carbon-X, a 14B open DNA model midtrained on DNA + text, claiming it beats every open language model and matches GPT-6.1 Sol at 1,000x lower cost. Stanford's Anshul Kundaje then ran a clinical-grade evaluation: 10 1k-token DNA snippets with pathogenic variants from GamowLabs' RareBench, 80 similar non-pathogenic variants, and 10 reference-genome snippets, testing direct pathogenicity prompting, phenotype explanation prompting, and mutation log-likelihoods. In 8/10 cases the model was worse than random chance (mean P(causal) 0.13, exactly the chance baseline) and confidently correct in only 1/10 — while a frontier LLM nails all of them. Kundaje says jointly embedding text and DNA is a great idea but this first attempt underperformed, and offered to collaborate.
More from Models
- User math: ~700M tokens daily quota forces mid-afternoon resets, 28-day event seen as a stopgap — yangyi · 2026-10-23
- Rumor: Anthropic training a model matching OpenAI's internal one; big launches in December — imjustnewatai · 2026-10-23
- AI researcher: Claude finally finished tedious research tasks I gave up on years ago — MikePFrank · 2026-10-23
- Dream Labs emerges from stealth with Dream-1 robot foundation model at $3/hour — scott_e_reed · 2026-10-23
- Claude Models on Google Cloud (Anthropic managed) is now generally available — claudeai · 2026-10-23
- Beam vs GLM 5.3 Abliterated: 10-round attack/defense test shows 96% of vulns patched — bigblueboo · 2026-10-23