Epoch AI tests whether agents can learn a board game from experience
Jsevillamol · x · 2026-07-21
- Epoch AI researcher Greg Burnham describes a benchmark built around the board game Earthborne Rangers to test whether AI can improve from experience.
- The game was chosen because it is playable solo and has relatively little visual or spatial reasoning.
- The team evaluates AI over five days of play. Out of the box, the systems play surprisingly well—often better than a human first-time player.
- But when asked to keep playing and take notes, performance stays flat: the agents do not appear to learn the game’s structure or improve over time.
- The point of the benchmark is to probe a gap between impressive first-turn competence and genuine experience-based learning.
More from AGI Musings
- Token quotas are reshaping how builders work, sleep, and recover — mobileraj · 2026-07-21
- NBER talk will present new evidence on how organizations use ChatGPT — daveholtz · 2026-07-21
- Writing for AI: When LLMs Become the New Audience for Online Content — IvyTatiana88 · 2026-07-21
- AI may push resistant tech workers toward union bargaining — Chobeat · 2026-07-21
- Jacob Tsimerman interview frames LLMs as a turning point for mathematical discovery — stevenstrogatz · 2026-07-21
- London is getting an AI-adjacent optimism picnic in Hyde Park on September 5 — isnit0 · 2026-07-21