Virtual Cell Benchmarks May Be Flawed
BoWang87 · x · 2026-07-08
A paper targeting ICML 2026 points out that standard perturbation benchmarks for virtual cell tasks might have severe issues.
The authors found that simple mean predictors outperform many models on these benchmarks. They argue the issue isn't poor model performance, but rather the evaluation method treating numerous irrelevant genes equally to a few signal genes, leading to distorted scores.
More from Research
- Kimi K3 may be strong on cyber, but token efficiency keeps it off UK AISIS — teortaxesTex · 2026-07-27
- ARC AGI 3 should have stayed private, with no examples or public dataset — flowersslop · 2026-07-27
- ExploitGym may have only 60–70% solvable tasks, fueling the OpenAI cheating debate — max_paperclips · 2026-07-27
- RTX 5090 local tests show Qwen Q6 can drop to 15 tok/s at 80k context — LFAdvice7984 · 2026-07-27
- Noahpinion quotes Chollet: intelligence may hit a hard ceiling — binarybits · 2026-07-27
- Paper argues graph topology can become the core operating system for AI agents — theomitsa · 2026-07-27