BioReason benchmarks 'seriously flawed': prompt leakage and set-selection confounds alleged
anshulkundaje · x · 2026-11-03
Anshul Kundaje continues his critique of BioReason: the benchmarks are seriously flawed with all kinds of prompt leakage, and even after apparent fixes by the Hugging Face team, positive/negative set selection likely contains many confounds.
More from Research
- EEG study: autism alters feedback evaluation, not raw perception — MacrinePhD · 2026-11-03
- Free guide by de Chaisemartin's team helps you pick the right DiD estimator — avicgoldfarb · 2026-11-03
- Interactive map charts Britain's AI for Science organisational landscape — ersatzben · 2026-11-03
- Human iEEG study supports predictive routing: same sound, different brain state — neuroprinciples · 2026-11-03
- Ant Group's SWE-Trail: history-guided trajectory collection yields better coding agents — antgroup · 2026-11-03
- BedBench: humans hit 93.3% on 3D color counting, best VLM just 7.2% — StanfordUniversity · 2026-11-03