Genomics researcher calls out popular benchmarks as opaque and flawed
anshulkundaje · x · 2026-09-03
Anshul Kundaje, a leading researcher in variant interpretation and genomics, notes that many popular benchmarks are highly flawed, and that collating disparate datasets to evaluate models introduces serious issues. He calls on benchmark makers to explicitly document how each benchmark was curated, QC'ed and vetted, along with its validity and limits.
Related event: Stanford Professor Says Most Public Benchmarks Have Systematic Flaws(3 posts)→
More from Research
- Skeptic dissects Astra's rumored recurrent architecture: likely just looping each layer twice — mike64_t · 2026-09-03
- Cosmos grantee to spend 3 months probing LLM reasoning faithfulness — hunarbatra · 2026-09-03
- Video Delta Net speeds up open video generation 75-90x: 14s video in 11s — xiuyu_l · 2026-09-03
- CoreAuto team publishes new blog post on neural architecture discovery — _arohan_ · 2026-09-03
- Researcher claims they may have 'cracked RL' for their task — andrew_n_carr · 2026-09-03
- Judea Pearl fires back in causal inference debate: RCT's new language deserves open scrutiny, not hiding — yudapearl · 2026-09-03