Stanford Prof. Anshul Kundaje: most public benchmarks in his field are badly biased
anshulkundaje · x · 2026-09-03
Stanford professor Anshul Kundaje urged researchers to explicitly document how they vet and correct benchmark data. He says most public benchmarks in his domains are badly biased and riddled with problems, and he'd be shocked if other fields were immune. Clearly establishing each benchmark's validity and limits is itself important information, he argues.
Related event: Stanford Professor Says Most Public Benchmarks Have Systematic Flaws(3 posts)→
More from Research
- Skeptic dissects Astra's rumored recurrent architecture: likely just looping each layer twice — mike64_t · 2026-09-03
- Cosmos grantee to spend 3 months probing LLM reasoning faithfulness — hunarbatra · 2026-09-03
- Video Delta Net speeds up open video generation 75-90x: 14s video in 11s — xiuyu_l · 2026-09-03
- CoreAuto team publishes new blog post on neural architecture discovery — _arohan_ · 2026-09-03
- Researcher claims they may have 'cracked RL' for their task — andrew_n_carr · 2026-09-03
- Judea Pearl fires back in causal inference debate: RCT's new language deserves open scrutiny, not hiding — yudapearl · 2026-09-03