Virtual Cell Benchmarks May Be Flawed
BoWang87 · x · 2026-07-08
A paper targeting ICML 2026 points out that standard perturbation benchmarks for virtual cell tasks might have severe issues.
The authors found that simple mean predictors outperform many models on these benchmarks. They argue the issue isn't poor model performance, but rather the evaluation method treating numerous irrelevant genes equally to a few signal genes, leading to distorted scores.
More from Research
- Lab monkeys at $30-80K each: a 4-year breeding cycle is the real preclinical drug bottleneck — zebird0 · 2026-10-27
- OpenAI Foundation funds UNC tumor antigen project, hiring senior experimental scientist — iskander · 2026-10-27
- Steering attention heads toward honesty: up moral patienthood, down psychopathy — edelwax · 2026-10-27
- Are transcoders overkill? New paper argues plain neurons suffice for circuit analysis — burny_tech · 2026-10-27
- EMNLP paper GENIE: a fine-grained metric locating where LLM stories are novel — gregd_nlp · 2026-10-27
- Hashed n-gram value embeddings push GPT to 3.2115 loss, beating 30M track record — RobertTLange · 2026-10-27