GPT-6 Astra Tested on MazeBench

Episode 1 · GPT-6 Astra Scores Just 14% After 60 Hours on 3D Spatial Benchmark MazeBench (2026-09-07, 8 posts)

Blogger patiencecave tested GPT-6 Astra on their self-built 3D open-world spatial reasoning benchmark MazeBench: the model ran for over 60 hours and scored just 14% without code execution, yet this is the best result on the benchmark to date—above GPT-5.6 Sol with Python (13%)—which the author reads as a generational capability leap, though far from passing MazeBench's harder challenges. Rohan Paul noted the score is roughly 7x that of comparison model Claude Fable 5.1. Reddit user WonderfulBuffalo32 posted a screenshot of GPT Astra scoring only 13% with no tools, arguing maze-type spatial reasoning lags far behind what official demos suggest.

Confirmed

• MazeBench is a long-horizon spatial reasoning benchmark: 100 gems hidden across 200+ rooms, requiring agents to rotate the view, manipulate boxes, recover from mistakes, and execute plans exceeding 100 steps; the no-Python track is harder since code-capable agents can solve levels in reverse.
• To save tokens, the eval allowed GPT-6 Astra to submit batched moves per turn; the model then consistently planned 10-20 steps ahead, cutting consumption from roughly 3 billion tokens to 350 million.
• Astra outperformed all prior agents, easily solving puzzles requiring 5 simultaneous moving parts (e.g., Klotski-style puzzles); exploration heatmaps show far broader coverage than Fable 5.1.
• Failure analysis shows Astra tends to interpret levels as top-down 2D planes, treating walls visible only from a 3D view as invisible barriers; it clears tutorial levels but fails 3D puzzles with more than 3 stacked objects.

Unconfirmed

• The Reddit post reports 13% for a no-tool run, slightly below patiencecave's 14%; whether the two evaluations used identical setups remains unverified.

Why it matters

Even the strongest new-generation models show structural weaknesses in long-horizon, true 3D spatial reasoning—"top-down map" world understanding remains a bottleneck. Meanwhile, batched planning's dramatic token reduction shows how much agent evaluation design affects cost.

• GPT-6 'Astra' burns 60+ hours in 3D spatial reasoning eval, scores just 14% — patience_cave · 2026-09-07
• GPT-6 Astra outperforms all prior agents, juggling 5 moving puzzle parts — patience_cave · 2026-09-07
• GPT-6 treats 3D levels as top-down maps, failing puzzles stacked over 3 blocks — patience_cave · 2026-09-07
• Batched moves let GPT-6 plan 10-20 steps ahead, cutting 3B tokens to 350M — patience_cave · 2026-09-07
• GPT-6 without code execution beats GPT-5.6 with Python on MazeBench — patience_cave · 2026-09-07
• GPT Astra scores just 13% on MazeBench without tools — Wonderful_Buffalo_32 · 2026-09-07
• GPT-6 Astra scores just 14% on MazeBench 3D spatial reasoning eval after 60+ hours — basedjensen · 2026-09-07
• GPT-6 Astra scores 14% on no-Python MazeBench, 7x Claude Fable 5.1's 2% — rohanpaul_ai · 2026-09-08

Episode 2 · GPT-6 Astra on MazeBench: Bare Model Beats GPT-5.6 With Tools (2026-09-14, 6 posts)

Blogger patiencecave systematically tested GPT-6 "Astra" on the self-built 3D spatial reasoning benchmark MazeBench. The core findings: code execution tools substantially boost the model's spatial reasoning, and Astra even without tools outperformed GPT-5.6 "Sol" running with tools.

Confirmed

• MazeBench is a maze benchmark testing visual-spatial reasoning in a 3D open world, created by patiencecave.
• GPT-6 Astra with Python code execution scored 23% after consuming 400 million tokens over 24 hours of reasoning; without Python it dropped to 14%.
• With code execution, Astra easily solved all entry-level levels with single thinking passes rarely exceeding 5 minutes; without tools, planning averaged 9 minutes and sometimes exceeded 20.
• The comparison shows Astra with code execution nearly doubled GPT-5.6 Sol with code execution; notably, Astra's bare score also beat the tooled GPT-5.6 Sol.

Why it matters

• Code tools drastically cut planning costs: per patiencecave's account, tool-based planning saved roughly half the cost for about a 60% gain.
• Astra's bare performance beating the previous generation with tools suggests a generational leap in GPT-6's "world model" and native spatial reasoning, a useful reference for evaluating new-model capabilities.

• GPT-6 Astra Scores Just 23% on MazeBench After 400M Tokens and 24 Hours of Reasoning — patience_cave · 2026-09-14
• MazeBench Recap: Astra With Code Execution Cuts Planning Time From 20+ Minutes to Under 5 — patience_cave · 2026-09-14
• Astra's 3D Planning: Code Execution Halves Cost for a 60% Gain Over Raw Reasoning — patience_cave · 2026-09-14
• GPT-6 Astra Nearly Doubles GPT-5.6 Sol on MazeBench, Wins Even Without Code Execution — patience_cave · 2026-09-14
• Maze Bench: GPT-6 Astra without code execution reportedly beats GPT-5.6 Sol with code — patience_cave · 2026-09-14
• MazeBench: 400M tokens and 24 hours of reasoning scores just 23% for GPT-6 Astra — mhmazur · 2026-09-14