Probe finds Claude Opus 5.5 memorized protein sequences; other LLMs show no such sign
anshulkundaje · x · 2026-11-18
Researchers released the LLM-bioseq-probe repo, testing via each model's API whether protein/DNA sequences were in training data. Key finding: Claude Fable 5.2 and Claude Opus 5.5 predict human and mouse coding sequence—but not the rest of the genome; they learned proteins, not genomes. No other model tested showed a clear sign.
The test asks models to complete the next amino acids of 668 bacterial/archaeal proteins from 167 "domain of unknown function" (DUF) Pfam families—little-studied, with no solved structures, almost no papers, and no UniProt sequence within 90% identity—so only training data could predict them. Each question is repeated with shuffled controls. Code, data, and full model replies are open-sourced.
More from Research
- FrontierCS 2: benchmark makes AI agents beat published CS research blind — arena · 2026-11-18
- Isomorphic Labs raising $5B led by Greenoaks and Sequoia as first cancer target nears trials — shiringhaffary · 2026-11-18
- Liquid AI's Maxime Labonne on post-training: SFT, RL, MOPD and regression — ziv_ravid · 2026-11-18
- Self-inoculation cuts RL reward hacking from 82% to 20% across 11 environments — repligate · 2026-11-18
- GPU QR competition audit: winning kernel ran 107x faster yet failed on new inputs — kfountou · 2026-11-18
- GPT-6.2 Astra tops Mercor's OpenWorldEngineer benchmark at 67.2% — morqon · 2026-11-18