Frontier Models Stumped by Simple Games: DiG-bench Tests AI Discovery
jcrwhittington · x · 2026-08-12
Researchers introduced DiG-bench, a new benchmark for evaluating the scientific discovery capabilities of LLMs. It features 70 text-based interactive games where models must experiment to uncover hidden rules and solve challenges.
By using purely text-based environments, the benchmark isolates discovery skills without visual confounds. Results show that while frontier models have improved, they still struggle with surprisingly simple problems that humans can beat on their first attempt.
Related event: DiG-bench: 70 Text Games Expose Frontier AI's Discovery Gaps(14 posts)→
More from Research
- DIY LEGO bench tests AI brick-building, GPT-6 Astra beats Opus — TheLegoless · 2026-10-16
- DIRECT: retrieval-based next-question suggestions for conversational agents — burkov · 2026-10-16
- AI-Driven Timing-Aware Power Optimization Uses Learned Ranking With Baseline-Preserving Repair — Amit Sharma · 2026-10-16
- Waypoint-OPD: Alibaba Gates On-Policy Distillation via Early-Late Teacher Shift Agreement — Bingxi Hou · 2026-10-16
- MJev-Spatial: Single-Pass Classification Beats Autoregressive VLMs on Spatial Reasoning — mindlab-research · 2026-10-16
- MLExpert: 35B MoE Agent Trained to Autonomously Handle Machine Learning Engineering — Shangheng Du · 2026-10-16