Nature Biotech paper: "single-cell models don't beat baselines" was a measurement problem
BoWang87 · x · 2026-10-06
Bo Wang's team publishes in Nature Biotechnology: "Deep Learning Perturbation Models Can Outperform Baselines on Calibrated Metrics".
- Several papers last year concluded DL perturbation models couldn't beat simple baselines, casting doubt on the Virtual Cell direction.
- The paper asks whether the metrics themselves can distinguish good from bad predictions—often they can't: MSE and control-referenced Pearson Δ frequently rank uninformative baselines above positive controls.
- With calibrated metrics (WMSE, weighted R²Δ, NIR), DL models do outperform mean, control, and linear baselines.
- Data matters too: Norman19 is 96% additive, giving models under 1% exposure to non-additive effects.
More from Research
- Huawei Noah's Tail-Influence Sampling Cuts CVaR Policy Evaluation MSE by Up to 76% — huawei-noah · 2026-10-06
- Google's KeyRec Achieves Best Long-Video VLM Results With Just 10% of Visual Token Budget — google · 2026-10-06
- 4DCodeBench Shows Frontier Models Reconstruct Static Scenes but Fail at Dynamics — 4DCodeBench · 2026-10-06
- OmniTaskonomy: Year-long study shows generation training can improve understanding tasks — WeijiaShi2 · 2026-10-06
- Newton proved the product rule without limits, using a discrete symmetric-difference trick — ctjlewis · 2026-10-06
- DeepMind's AI designs enzymes from scratch: 99x drug building block yield, plastic-eating at 90°C — 141_1337 · 2026-10-06