EurekaBench crowdsources expert-written problems to test AI scientific insight
udmrzn · x · 2026-10-15
EurekaBench, built by Jiayi Geng and colleagues with domain experts, asks whether AI scientist agents can go beyond trial-and-error optimization — echoing Terence Tao's view of science as a "lighthouse" of insight — and discover genuinely novel scientific insights through experimentation.
The benchmark spans 6 science domains and is now open for expert contributions:
- Additional insight questions: different approaches to the same problem yield different insights; contributors who solved included problems differently can add what agents could discover.
- New scientific problems: published-but-partially-solved or unpublished discovery problems, both in existing domains and new ones.
Interested experts can email [email protected] or use the contribution form.
More from Research
- RFD4-Proteina's 'underrated' docs detail partial diffusion for protein redesign — AllThingsApx · 2026-10-23
- MPI-Sec PhD Thesis on Vulnerability Detection Benchmarks Includes Pre-LLM Classic "Top Score on the Wrong Exam" — mboehme_ · 2026-10-23
- LiveNerf's 29-day data finds no nerfing of Claude Opus 5.5 — TheOnlyVibemaster · 2026-10-23
- New paper makes 9 falsifiable predictions toward a major AI-enabled terror attack by 2027-2028 — StephenLCasper · 2026-10-23
- HumanoidPose recovers humanoid robots' full-body joint states from RGB video alone — cholz · 2026-10-23
- Sophont's Holon-S1 predicts genes and biomarkers from routine pathology slides — iScienceLuvr · 2026-10-23