Stanford's Kundaje calls out Arc virtual cell challenge for confounding data advantage with method quality
anshulkundaje · x · 2026-11-20
Stanford professor Anshul Kundaje publicly criticized the design of the Arc Institute virtual cell prediction challenge, arguing the winning team's undisclosed data-access advantage totally confounds data advantage with methodological superiority — so the challenge can't demonstrate method quality at all.
- The challenged team responded that test cell lines were deliberately picked: one reasonably close, one partial, one never seen before, with Illumina single-cell sequencing on a completely different technology platform.
- They also cite extensive ablation testing (removing closest N matches) showing graceful performance degradation as novelty increases.
- Kundaje stresses this isn't a critique of the team but a fundamental flaw in challenge design, questioning what the challenge is even testing.
A substantive academic dispute over benchmark fairness in AI for Science.
More from Fun
- GTA VI hacked to 60fps with DLSS 5 and first-person within 48 hours, footage looks lifelike — LinusEkenstam · 2026-11-20
- Gemini 4 Neon called the world's best at naming things — willcb · 2026-11-20
- Palmer Luckey spotted using a Samsung phone at NYT DealBook Summit — alexmacgregor__ · 2026-11-20
- Train a real LLM entirely in your browser with this interactive demo — morgymcg · 2026-11-20
- Omarchy Linux community spreads from Shenzhen to Singapore, now running on Xiaomi Pad — juntao · 2026-11-20
- Mitchell Hashimoto: What's Wealth For If Not Making Terminals Great — itsOmSarraf_ · 2026-11-20