Hugging Face Open-Sources Carbon-X, a 14B DNA-Language Model Rivaling GPT-6.1 Sol at 1/1000 the Cost

Hugging Face's biology team (Loubna Ben Allal et al.) open-sourced the DNA language model Carbon-X on October 22: a 14B-parameter open model that, for the first time, unifies DNA sequences and natural language in a single model. A blog post and model weights have been released, and it can run locally. The team calls it the strongest open DNA model to date, with the open release intended to let anyone study its capabilities and limitations and advance open science in AI for Biology.

Confirmed

• How it was built: starting from Qwen3-14B, midtraining (continued pretraining) was performed on 300B tokens of DNA, English, and interleaved data; a hybrid tokenizer is used — BPE for English and 6-mer tokenization for DNA — enabling a single model to handle both modalities, answer questions about sequence function and mutation effects, and directly generate new DNA.
• Performance: it beats all tested open-source language models on five DNA understanding and generation benchmarks, averaging 66.7, a 27-point average gain over the base Qwen3-14B, matching GPT-6.1 Sol at over 1000x lower inference cost (also stated as roughly 1/1800); its strongest areas are mutation effect prediction and DNA sequence completion.
• Training methodology: applying pure RL directly to Qwen3-14B never broke past the 50% random baseline on FrontierVEP; RL from a midtraining checkpoint got past random and kept improving, leading the team to conclude midtraining is key.
• Agent usage: after Kimi K3 could consult Carbon-X as a sub-agent, its DNA intelligence index rose from 49.4% to 60.5%, an 11.1-point gain across the five benchmarks.
• Frontier model refusals: hands-on testing showed Opus 5.5's score on next-base prediction dropping from 17.7% at low reasoning effort to 1.1% at the highest, because stronger reasoning increases refusal rates; Gemini 4 and others show similar trends, which the team argues hampers legitimate genomics research.

Why it matters

Carbon-X matches frontier closed-source models on DNA tasks at extremely low cost, validating that language models can gain substantially from DNA training and demonstrating the viability of midtraining + a hybrid tokenizer; the excessive-refusal problem it uncovered also motivates open alternatives for biological research.

2026-10-22 ~ 2026-10-22 · 12 related posts

• [source] Hugging Face releases open DNA model Carbon-X, matching GPT-6.1 Sol at 1/1800 the cost — LoubnaBenAllal1 · 2026-10-22
• Carbon-X averages 66.7 across five DNA benchmarks, closing in on GPT-6.1 Sol — LoubnaBenAllal1 · 2026-10-22
• Carbon-X beats all open LLMs on five DNA benchmarks, +27 points over Qwen3-14B — LoubnaBenAllal1 · 2026-10-22
• [source] How Carbon-X was built: hybrid BPE + 6-mer tokenizer over 300B DNA-text tokens — LoubnaBenAllal1 · 2026-10-22
• RL alone stalls at chance: Carbon-X shows midtraining unlocks gains RL can't — LoubnaBenAllal1 · 2026-10-22
• Carbon-X as a sub-agent lifts Kimi K3's DNA intelligence index by 11.1 points — LoubnaBenAllal1 · 2026-10-22
• Open-sourced Carbon-X DNA model exposes frontier LLM refusals: Opus 5.5 drops 17.7% to 1.1% on genomics — LoubnaBenAllal1 · 2026-10-22
• Frontier models refuse more genomics requests as reasoning effort rises, Carbon-X team finds — LoubnaBenAllal1 · 2026-10-22
• [source] Carbon-X: Open 14B DNA Language Model Beats Open Rivals, Runs Locally via llama.cpp — loubnabnl · 2026-10-22

3 near-duplicate retellings: lvwerra · burny_tech · MaziyarPanahi