Nature Biotech: calibrated metrics show deep learning perturbation models can beat baselines
VectorInst · x · 2026-10-02
A Brief Communication in Nature Biotechnology revisits the claim that deep learning-based genetic perturbation models fail to beat uninformative baselines.
- The authors introduce a positive-control baseline and a metric calibration framework, evaluated across 14 datasets and 18 metrics.
- Common benchmarking metrics (e.g., MSE, Pearson Δ) are frequently miscalibrated, with reduced sensitivity to genuine model performance.
- Under well-calibrated metrics, deep-learning perturbation models can indeed outperform uninformative baselines.
The paper serves as a methodological correction to an earlier 2025 report concluding that these models don't beat simple linear baselines.
Related event: Calibrated Metrics Vindicate Deep Learning Models of Genetic Perturbation(3 posts)→
More from Research
- Local Support Learning: 7B LLMs learn new tasks at full capacity without forgetting or old data — CatAstro_Piyush · 2026-10-02
- MIT researchers unveil interface exposing LLM internals during chatbot personality design — patpat_mit · 2026-10-02
- Fourth UK AI Conference Proceedings Now Live on PMLR as Volume 348 — lawrennd · 2026-10-02
- New paper: Training-time internal signals can improve alignment without hurting white-box monitoring — jonasgeiping · 2026-10-02
- Reka Open-Sources RIDM, Extracting Camera and Motor Commands from Raw Video — RekaAILabs · 2026-10-02
- YC Paper Club Explores AI Compute Beyond GPUs: Optical, Neuromorphic and Biological Computing — ycombinator · 2026-10-02