Nature Biotech: calibrated metrics show deep learning perturbation models can beat baselines
VectorInst · x · 2026-10-02
A Brief Communication in Nature Biotechnology revisits the claim that deep learning-based genetic perturbation models fail to beat uninformative baselines.
- The authors introduce a positive-control baseline and a metric calibration framework, evaluated across 14 datasets and 18 metrics.
- Common benchmarking metrics (e.g., MSE, Pearson Δ) are frequently miscalibrated, with reduced sensitivity to genuine model performance.
- Under well-calibrated metrics, deep-learning perturbation models can indeed outperform uninformative baselines.
The paper serves as a methodological correction to an earlier 2025 report concluding that these models don't beat simple linear baselines.
Related event: Calibrated Metrics Vindicate Deep Learning Models of Genetic Perturbation(3 posts)→
More from Research
- Ramp releases SWE-Bench v2: 500 production-built tasks, 6.2 Astra tops at 76.4% — AniBaddepudi · 2026-11-19
- Stanford's Le Cong to talk AI co-scientists for RNA design at 14th mRNA Health Conference — MengdiWang10 · 2026-11-19
- New paper shows DiT latents can be linearly probed to predict pixels, skipping the VAE decoder — kwangmoo_yi · 2026-11-19
- SAIL 2027 opens abstract submissions for AI in clinical care, deadline Jan 15 — zakkohane · 2026-11-19
- The academia LLM paradox: everyone writes with AI, nobody wants to read or be judged by it — wandedob · 2026-11-19
- Baseten hosts All-Gather, its first research conference on where AI's next gains will come from — baseten · 2026-11-19