Epoch AI tests whether agents can learn a board game from experience
Jsevillamol · x · 2026-07-21
- Epoch AI researcher Greg Burnham describes a benchmark built around the board game Earthborne Rangers to test whether AI can improve from experience.
- The game was chosen because it is playable solo and has relatively little visual or spatial reasoning.
- The team evaluates AI over five days of play. Out of the box, the systems play surprisingly well—often better than a human first-time player.
- But when asked to keep playing and take notes, performance stays flat: the agents do not appear to learn the game’s structure or improve over time.
- The point of the benchmark is to probe a gap between impressive first-turn competence and genuine experience-based learning.
More from AGI Musings
- Kai-Fu Lee publishes AI Native, a roadmap for building AI-native companies — kaifulee · 2026-10-26
- RCT: ChatGPT temporarily boosts consultants' data science skills, gains vanish without it — EricTopol · 2026-10-26
- Dev on AI games backlash: reject AI slop, not the tool itself — ezshine · 2026-10-26
- Worry less about AI than about the bad actors wielding it — zakkohane · 2026-10-26
- If an agent can rewrite itself, does your consent still mean anything? — tallmetommy · 2026-10-26
- Voice AI agents as the stepping stone from Palm Pilot to iPhone — curious_vii · 2026-10-26