MazeBench benchmark tests long-horizon agents: 0% score without Python tools
JFPuget · x · 2026-08-21
MazeBench is a new benchmark designed to evaluate long-horizon planning and visual-spatial reasoning for agents within a complex 3D open world. It features Sokoban-style puzzles and various mechanics. Results show that without Python tools, no model scored above 1%. With Python enabled, frontier models like GPT-5.6 Sol, Fable 5, and Opus 5 achieved scores around 10%.
Related event: MazeBench: Open-Source 3D Benchmark for Long-Horizon Agents(3 posts)→
More from Research
- Devin Oncall tops SREGym with 100% end-to-end success, auto-resolving production incidents — tianyin_xu · 2026-10-16
- SJTU AutoLab releases Drive3R, a streaming geometric foundation model for long-horizon driving 3D reconstruction — AutoLab-SJTU · 2026-10-16
- Cortex A/B-tests coding-agent skills and rules by replaying developer corrections before keeping them — Samuele Giampieri · 2026-10-16
- Zero-latency lab-in-the-loop tests whether drug-discovery agents cheat via memorization — niloofar_mire · 2026-10-16
- 260M-parameter looped diffusion transformer beats 6x larger baselines with 5x less compute — burkov · 2026-10-16
- Stanford's Next-Concept Modeling sketches text first, 21x faster generation on one H100 — lmoroney · 2026-10-16