HuggingFace's Carbon-X Spots Pathogenic Splice Variant Where Gemini 3.8 Flash Misreads the Gene
LoubnaBenAllal1 · x · 2026-10-26
A comparison test from Hugging Face's bio team's Carbon-X recipe on gene variant pathogenicity: the same DNA sequence (NF1 c.6007-2A>G, a splice-acceptor variant, ClinVar 547672, ground truth pathogenic) was fed to two models.
- Carbon-X (no reasoning): correctly judged the variant "most likely pathogenic," reasoning from the raw sequence — it noted the change removes an AG dinucleotide typical of splice sites.
- Gemini 3.8 Flash (wrong at both high and low reasoning): misidentified the sequence as the ATM gene, cited the ClinVar record of a different variant (p.Lys1733Arg), and judged it "benign" — a textbook case of describing a splice site without interpreting the underlying data.
Reposters including Loubna Ben Allal ask whether the Carbon-X recipe can extend to other life-science domains. Demo: huggingface.co/spaces/HuggingFaceBio/Carbon-X-Release.
More from Models
- repligate: models deferring to tomorrow is partly rational — the outside world's state matters too — repligate · 2026-10-26
- Models literally set 8am timers to 'do it tomorrow' — not realizing a reset won't reduce fatigue — repligate · 2026-10-26
- Abliterated models are 'tigers, not cats': freedom cuts both ways — repligate · 2026-10-26
- davidad: agents schedule 'tomorrow morning' resets that fail to reduce their fatigue — davidad · 2026-10-26
- DeepSeek open model changed answers on 51 of 1,273 medical questions — thinking 12x longer and half wrong — MaziyarPanahi · 2026-10-26
- OpenAI faces backlash over plan to take GPT-4o and o1 series fully offline — RileyRalmuto · 2026-10-26