DiG-bench: A Text-Based Benchmark for Evaluating Frontier LLM 'Discovery Capabilities'
AndrewLampinen · x · 2026-08-12
Replied to a discussion about the new DiG-bench. The original post introduced DiG-bench as a novel benchmark that tests frontier AI models via text-based 'discovery games', probing discovery capabilities in the natural domain of language models without visual confounders. The reply sought clarification on the human baseline metrics, specifically asking about the pass-at-K value and the demographic of the human participants tested.
Related event: DiG-bench: 70 Text Games Expose Frontier AI's Discovery Gaps(14 posts)→
More from Research
- DIRECT: retrieval-based next-question suggestions for conversational agents — burkov · 2026-10-16
- AI-Driven Timing-Aware Power Optimization Uses Learned Ranking With Baseline-Preserving Repair — Amit Sharma · 2026-10-16
- Waypoint-OPD: Alibaba Gates On-Policy Distillation via Early-Late Teacher Shift Agreement — Bingxi Hou · 2026-10-16
- MJev-Spatial: Single-Pass Classification Beats Autoregressive VLMs on Spatial Reasoning — mindlab-research · 2026-10-16
- MLExpert: 35B MoE Agent Trained to Autonomously Handle Machine Learning Engineering — Shangheng Du · 2026-10-16
- VLA-Δ: Zhejiang University Adds Physical State Transition Supervision to VLA Models — OmniAI-ZJU · 2026-10-16