DiG-bench: 70 Text-Based Discovery Games to Test AI's Scientific Exploration
jcrwhittington · x · 2026-08-12
To ensure AI can genuinely make new scientific discoveries rather than just combining known tools, researchers introduced DiG-bench, a benchmark of 70 handcrafted text-based discovery games.
- Gameplay: Each game is a self-contained world with hidden rules and win conditions, requiring real experimentation and exploration to beat.
- Difficulty: Features seven difficulty tiers calibrated to frontier models. The games are challenging but beatable, with at least one human beating every game on their first try.
- Data Split: 21 games are public, while 49 remain private for secure and reliable evaluation.
Related event: DiG-bench: 70 Text Games Expose Frontier AI's Discovery Gaps(14 posts)→
More from Research
- DIY LEGO bench tests AI brick-building, GPT-6 Astra beats Opus — TheLegoless · 2026-10-16
- DIRECT: retrieval-based next-question suggestions for conversational agents — burkov · 2026-10-16
- AI-Driven Timing-Aware Power Optimization Uses Learned Ranking With Baseline-Preserving Repair — Amit Sharma · 2026-10-16
- Waypoint-OPD: Alibaba Gates On-Policy Distillation via Early-Late Teacher Shift Agreement — Bingxi Hou · 2026-10-16
- MJev-Spatial: Single-Pass Classification Beats Autoregressive VLMs on Spatial Reasoning — mindlab-research · 2026-10-16
- MLExpert: 35B MoE Agent Trained to Autonomously Handle Machine Learning Engineering — Shangheng Du · 2026-10-16