Frontier models flunk 1,000-variant DNA classification: refusals, token blowouts, encrypted reasoning
_lewtun · x · 2026-11-03
While building FineDNA's SFT and RL subsets, Lewis Tunstall tested three frontier models on classifying 1,000 genetic variants as harmless or harmful. GPT-6.1 Sol answered with bare labels (reasoning encrypted), Gemini 4 Argon refused over 50% of the time, and Opus 5.5 ran out of tokens on 20% of prompts. Open-weight models were barely better than random — almost nothing to distill.
More from Models
- Sage 2 extraction model tops Extract Bench with 77.8% field accuracy, beating Cloudflare's Clairic — iamrobotbear · 2026-11-03
- Startup launches Sage2, a streaming decision model claimed 166x cheaper than Opus 5.5 — mhdfaran · 2026-11-03
- Researcher Predicts Decision Models Will Explode as Frontier Labs Post-Train With Them — marktenenholtz · 2026-11-03
- Leaked Mistral Large 4 abstract: 1T-param MoE with 52B active, fits on 4 GB300 GPUs — qtnx_ · 2026-11-03
- OpenAI's new LifeVal benchmark reportedly tests everyday tasks; GPT-6 Astra hits 100% on several — mark_k · 2026-11-03
- 36 runs, 12 open models, one medical exam question — and all disagree with the answer key — MaziyarPanahi · 2026-11-03