MazeBench benchmark tests long-horizon agents: 0% score without Python tools
JFPuget · x · 2026-08-21
MazeBench is a new benchmark designed to evaluate long-horizon planning and visual-spatial reasoning for agents within a complex 3D open world. It features Sokoban-style puzzles and various mechanics. Results show that without Python tools, no model scored above 1%. With Python enabled, frontier models like GPT-5.6 Sol, Fable 5, and Opus 5 achieved scores around 10%.
Related event: MazeBench launches as open long-horizon agent benchmark(2 posts)→
More from Research
- Long-Horizon Agents: From Seconds-Long Tasks to Full Workday Autonomy — agihouse_org · 2026-08-21
- QuoteBench: matched scores can hide command-path parsing failures in coding agents — Shangao Li · 2026-08-21
- Lightwheel open-sources EgoSuite-Open100K: 100K hours of egocentric video for robot hands — lukas_m_ziegler · 2026-08-21
- Long-Horizon Agents: Evolution from Seconds to Hours of Autonomous Work — agihouse_org · 2026-08-21
- poolside researchers on turning tens of thousands of data-mix experiments into frontier-moving models — arena · 2026-08-21
- Google open-sources TIPSv2 image-text encoders, SOTA on all four zero-shot segmentation benchmarks — bdsqlsz · 2026-08-21