Peer-reviewed paper questions whether gene expression model benchmarks are informative
simocristea · x · 2026-10-02
A newly peer-reviewed paper formalizes a pointed question: are the benchmarks used to evaluate foundation models on gene expression data actually informative? Common evaluations rely on arithmetic-mean baselines averaged over thousands of genes, most irrelevant to the phenotype of interest — largely because real measurements of the target phenotype are often unavailable. A systematic critique of benchmark design for bio foundation models.
Related event: Calibrated Metrics Vindicate Deep Learning Models of Genetic Perturbation(3 posts)→
More from Research
- LEV: Looped decision model lifts 2.6B accuracy 63.2% to 76.2% with adaptive compute — jiqizhixin · 2026-11-19
- Aurora2-400M-A20M: a sparse MoE pretrained from scratch in 36 hours on one RTX Pro 6000 — Tall_Abrocoma_3533 · 2026-11-19
- Researcher calls METR/Anthropic '2x speedup' RSI analysis statistically dubious — jessi_cata · 2026-11-19
- Linear Probes Could Reveal Which Persona a Base Model Is Playing, Researchers Suggest — cephaloform · 2026-11-19
- ETH's Anymal robot completes 10km autonomous alpine hike with VLM-guided ROAM system — ChongZzZhang · 2026-11-19
- OpenVax codebase grows from 78K to 565K lines in 15 months — iskander · 2026-11-19