Developer Pushes Back: Humanity's Last Exam Dataset Has Very Low Error Rate
Bayesian0_0 · x · 2026-09-02
In response to claims that the Humanity's Last Exam dataset contains many errors and suffers from poor methodology, developer Sauers pushed back, saying he personally checked many of the biology questions and found possibly only one that was debatable. He argued that the accuracy statistics are informative and that the related analysis article should not simply be deleted.
More from Research
- Opus-assisted open design: a $31k low-field head MRI sketched in hours — DreamFly_13 · 2026-10-18
- semiconductor.diy: open-source guides to make real chips in a garden shed — jwt0625 · 2026-10-18
- Work in progress: geometric structure of GPG projections as Markov chain coarsening — michaelchchoi · 2026-10-18
- Backprop-free pretraining gets 2.3x speedup as Noise method crushes Dust record on 8 GPUs — inductionheads · 2026-10-18
- q3dviewer: open-source Qt+OpenGL tool for painter-like point cloud editing — 4310sy · 2026-10-18
- Smarter Every Day's Destin to defend PhD dissertation on impact fusion — CatAstro_Piyush · 2026-10-18