billion-context: open-source gene expression tokenizer hits 92.4% accuracy on cell-type classification
udmrzn · x · 2026-11-10
The research team released billion-context (5.7k GitHub stars), which converts gene expression profiles into token sequences for pretraining, achieving 92.4% accuracy across 34 cell types on an independent test set.
- Core idea: encode single-cell RNA-seq expression values as discrete tokens sorted by expression level, modeled with a standard LLM architecture
- Data scale: 36M+ human and mouse cells across 28 organs, all from public datasets
- Results: outperforms Scimilarity and CellTypist by 7.3 F1 points on average for cross-organ zero-shot cell-type annotation
- Open artifacts: training code, preprocessing pipeline, and two model sizes (120M and 650M params) with full reproduction docs
- Limitations: systematic bias on samples with mitochondrial overexpression, and high inference cost (0.4s per cell)
A demonstration that "treating biology as language" works in single-cell genomics.
More from AGI Musings
- AI chatbot referrals surge but can't offset lost search traffic, Axios reports — gaganghotra_ · 2026-11-10
- RSI is just an engineering problem — ASI is a different story entirely — haider1 · 2026-11-10
- Claude-built visualization: insects may hold 62 million times more experience-years than humans — javirandor · 2026-11-10
- Academia vs Industry: The Salary Gap for AI Scientists — RexDouglass · 2026-11-10
- Hand-written research abstract flagged 100% AI by detectors — Reehbeeh-Turnip-223 · 2026-11-10
- Avalon explores how AI agents learn to organise and collaborate autonomously — rosstaylor90 · 2026-11-10