GPT-6 Astra Tested on MazeBench

Episode 1 · GPT-6 Astra Scores Just 14% After 60 Hours on 3D Spatial Benchmark MazeBench (2026-09-07, 8 posts)

Blogger patiencecave tested GPT-6 Astra on their self-built 3D open-world spatial reasoning benchmark MazeBench: the model ran for over 60 hours and scored just 14% without code execution, yet this is the best result on the benchmark to date—above GPT-5.6 Sol with Python (13%)—which the author reads as a generational capability leap, though far from passing MazeBench's harder challenges. Rohan Paul noted the score is roughly 7x that of comparison model Claude Fable 5.1. Reddit user WonderfulBuffalo32 posted a screenshot of GPT Astra scoring only 13% with no tools, arguing maze-type spatial reasoning lags far behind what official demos suggest.

Confirmed

• MazeBench is a long-horizon spatial reasoning benchmark: 100 gems hidden across 200+ rooms, requiring agents to rotate the view, manipulate boxes, recover from mistakes, and execute plans exceeding 100 steps; the no-Python track is harder since code-capable agents can solve levels in reverse.
• To save tokens, the eval allowed GPT-6 Astra to submit batched moves per turn; the model then consistently planned 10-20 steps ahead, cutting consumption from roughly 3 billion tokens to 350 million.
• Astra outperformed all prior agents, easily solving puzzles requiring 5 simultaneous moving parts (e.g., Klotski-style puzzles); exploration heatmaps show far broader coverage than Fable 5.1.
• Failure analysis shows Astra tends to interpret levels as top-down 2D planes, treating walls visible only from a 3D view as invisible barriers; it clears tutorial levels but fails 3D puzzles with more than 3 stacked objects.

Unconfirmed

• The Reddit post reports 13% for a no-tool run, slightly below patiencecave's 14%; whether the two evaluations used identical setups remains unverified.

Why it matters

Even the strongest new-generation models show structural weaknesses in long-horizon, true 3D spatial reasoning—"top-down map" world understanding remains a bottleneck. Meanwhile, batched planning's dramatic token reduction shows how much agent evaluation design affects cost.

• GPT-6 'Astra' burns 60+ hours in 3D spatial reasoning eval, scores just 14% — patience_cave · 2026-09-07
• GPT-6 Astra outperforms all prior agents, juggling 5 moving puzzle parts — patience_cave · 2026-09-07
• GPT-6 treats 3D levels as top-down maps, failing puzzles stacked over 3 blocks — patience_cave · 2026-09-07
• Batched moves let GPT-6 plan 10-20 steps ahead, cutting 3B tokens to 350M — patience_cave · 2026-09-07
• GPT-6 without code execution beats GPT-5.6 with Python on MazeBench — patience_cave · 2026-09-07
• GPT Astra scores just 13% on MazeBench without tools — Wonderful_Buffalo_32 · 2026-09-07
• GPT-6 Astra scores just 14% on MazeBench 3D spatial reasoning eval after 60+ hours — basedjensen · 2026-09-07
• GPT-6 Astra scores 14% on no-Python MazeBench, 7x Claude Fable 5.1's 2% — rohanpaul_ai · 2026-09-08

Episode 2 · MazeBench Tests GPT-6 Astra: Code Execution Dramatically Boosts 3D Spatial Reasoning (2026-09-14, 6 posts)

Blogger patiencecave ran a systematic evaluation of GPT-6 "Astra" on their own 3D spatial reasoning benchmark, MazeBench. The core conclusion: code execution tools significantly boost the model's spatial reasoning, and Astra without tools still beats the previous-generation GPT-5.6 "Sol" with tools.

Confirmed

• MazeBench is a maze benchmark testing visuospatial reasoning in 3D open worlds, created by patiencecave.
• With Python code execution, GPT-6 Astra scored 23% after consuming 400 million tokens over 24 hours of reasoning; without Python, its score dropped to 14%.
• With code execution, Astra easily solved all beginner levels, with a single train of thought rarely exceeding 5 minutes; without tools, planning averaged 9 minutes per attempt, sometimes over 20 minutes.
• Cross-comparison: Astra with code execution nearly doubled GPT-5.6 Sol with code execution; more notably, Astra's bare run without code execution also beat GPT-5.6 Sol with tools.

Why it matters

• Results show code tools can dramatically cut planning costs: per patiencecave's account, planning with tools saves roughly half the cost for roughly a 60% improvement.
• Astra's bare run beating the previous generation with tools indicates a generational leap in GPT-6's "world model" and native spatial reasoning, useful reference for assessing the true capabilities of new-generation models.

• GPT-6 Astra Scores Just 23% on MazeBench After 400M Tokens and 24 Hours of Reasoning — patience_cave · 2026-09-14
• MazeBench Recap: Astra With Code Execution Cuts Planning Time From 20+ Minutes to Under 5 — patience_cave · 2026-09-14
• Astra's 3D Planning: Code Execution Halves Cost for a 60% Gain Over Raw Reasoning — patience_cave · 2026-09-14
• GPT-6 Astra Nearly Doubles GPT-5.6 Sol on MazeBench, Wins Even Without Code Execution — patience_cave · 2026-09-14
• Maze Bench: GPT-6 Astra without code execution reportedly beats GPT-5.6 Sol with code — patience_cave · 2026-09-14
• MazeBench: 400M tokens and 24 hours of reasoning scores just 23% for GPT-6 Astra — mhmazur · 2026-09-14