GPT-6 Astra on MazeBench: Bare Model Beats GPT-5.6 With Tools
Blogger patiencecave systematically tested GPT-6 "Astra" on the self-built 3D spatial reasoning benchmark MazeBench. The core findings: code execution tools substantially boost the model's spatial reasoning, and Astra even without tools outperformed GPT-5.6 "Sol" running with tools.
Confirmed
- MazeBench is a maze benchmark testing visual-spatial reasoning in a 3D open world, created by patiencecave.
- GPT-6 Astra with Python code execution scored 23% after consuming 400 million tokens over 24 hours of reasoning; without Python it dropped to 14%.
- With code execution, Astra easily solved all entry-level levels with single thinking passes rarely exceeding 5 minutes; without tools, planning averaged 9 minutes and sometimes exceeded 20.
- The comparison shows Astra with code execution nearly doubled GPT-5.6 Sol with code execution; notably, Astra's bare score also beat the tooled GPT-5.6 Sol.
Why it matters
- Code tools drastically cut planning costs: per patiencecave's account, tool-based planning saved roughly half the cost for about a 60% gain.
- Astra's bare performance beating the previous generation with tools suggests a generational leap in GPT-6's "world model" and native spatial reasoning, a useful reference for evaluating new-model capabilities.
2026-09-14 ~ 2026-09-14 · 6 related posts
- Episode 1: GPT-6 Astra Scores Just 14% After 60 Hours on 3D Spatial Benchmark MazeBench(2026-09-07, 8 posts)
- Episode 2: GPT-6 Astra on MazeBench: Bare Model Beats GPT-5.6 With Tools(2026-09-14, 6 posts)
Primary sources
- [source] GPT-6 Astra Scores Just 23% on MazeBench After 400M Tokens and 24 Hours of Reasoning — patience_cave · 2026-09-14
- MazeBench Recap: Astra With Code Execution Cuts Planning Time From 20+ Minutes to Under 5 — patience_cave · 2026-09-14
- Astra's 3D Planning: Code Execution Halves Cost for a 60% Gain Over Raw Reasoning — patience_cave · 2026-09-14
- [source] GPT-6 Astra Nearly Doubles GPT-5.6 Sol on MazeBench, Wins Even Without Code Execution — patience_cave · 2026-09-14
- Maze Bench: GPT-6 Astra without code execution reportedly beats GPT-5.6 Sol with code — patience_cave · 2026-09-14
1 near-duplicate retellings: mhmazur