MazeBench Tests GPT-6 Astra: Code Execution Dramatically Boosts 3D Spatial Reasoning
Blogger patiencecave ran a systematic evaluation of GPT-6 "Astra" on their own 3D spatial reasoning benchmark, MazeBench. The core conclusion: code execution tools significantly boost the model's spatial reasoning, and Astra without tools still beats the previous-generation GPT-5.6 "Sol" with tools.
Confirmed
- MazeBench is a maze benchmark testing visuospatial reasoning in 3D open worlds, created by patiencecave.
- With Python code execution, GPT-6 Astra scored 23% after consuming 400 million tokens over 24 hours of reasoning; without Python, its score dropped to 14%.
- With code execution, Astra easily solved all beginner levels, with a single train of thought rarely exceeding 5 minutes; without tools, planning averaged 9 minutes per attempt, sometimes over 20 minutes.
- Cross-comparison: Astra with code execution nearly doubled GPT-5.6 Sol with code execution; more notably, Astra's bare run without code execution also beat GPT-5.6 Sol with tools.
Why it matters
- Results show code tools can dramatically cut planning costs: per patiencecave's account, planning with tools saves roughly half the cost for roughly a 60% improvement.
- Astra's bare run beating the previous generation with tools indicates a generational leap in GPT-6's "world model" and native spatial reasoning, useful reference for assessing the true capabilities of new-generation models.
2026-09-14 ~ 2026-09-14 · 6 related posts
- Episode 1: GPT-6 Astra Scores Just 14% After 60 Hours on 3D Spatial Benchmark MazeBench(2026-09-07, 8 posts)
- Episode 2: MazeBench Tests GPT-6 Astra: Code Execution Dramatically Boosts 3D Spatial Reasoning(2026-09-14, 6 posts)
Primary sources
- GPT-6 Astra Scores Just 23% on MazeBench After 400M Tokens and 24 Hours of Reasoning — patience_cave ·
- Astra's 3D Planning: Code Execution Halves Cost for a 60% Gain Over Raw Reasoning — patience_cave ·
- GPT-6 Astra Nearly Doubles GPT-5.6 Sol on MazeBench, Wins Even Without Code Execution — patience_cave ·
- [source] GPT-6 Astra Scores Just 23% on MazeBench After 400M Tokens and 24 Hours of Reasoning — patience_cave · 2026-09-14
- MazeBench Recap: Astra With Code Execution Cuts Planning Time From 20+ Minutes to Under 5 — patience_cave · 2026-09-14
- [source] Astra's 3D Planning: Code Execution Halves Cost for a 60% Gain Over Raw Reasoning — patience_cave · 2026-09-14
- [source] GPT-6 Astra Nearly Doubles GPT-5.6 Sol on MazeBench, Wins Even Without Code Execution — patience_cave · 2026-09-14
- Maze Bench: GPT-6 Astra without code execution reportedly beats GPT-5.6 Sol with code — patience_cave · 2026-09-14
1 near-duplicate retellings: mhmazur