Expert: many popular genomics benchmarks are deeply flawed — publish their limits
anshulkundaje · x · 2026-09-03
Anshul Kundaje says he can cite many examples of highly flawed popular benchmarks in variant interpretation and genomics, plus issues with collating disparate datasets for model evaluation. Establishing and clearly exposing the validity and limits of every benchmark matters, and he hopes such information will be explicitly provided.
Related event: Stanford Professor Says Most Public Benchmarks Have Systematic Flaws(3 posts)→
More from Research
- Skeptic dissects Astra's rumored recurrent architecture: likely just looping each layer twice — mike64_t · 2026-09-03
- Cosmos grantee to spend 3 months probing LLM reasoning faithfulness — hunarbatra · 2026-09-03
- Video Delta Net speeds up open video generation 75-90x: 14s video in 11s — xiuyu_l · 2026-09-03
- CoreAuto team publishes new blog post on neural architecture discovery — _arohan_ · 2026-09-03
- Researcher claims they may have 'cracked RL' for their task — andrew_n_carr · 2026-09-03
- Judea Pearl fires back in causal inference debate: RCT's new language deserves open scrutiny, not hiding — yudapearl · 2026-09-03