GPT-6 Astra Scores Just 14% After 60 Hours on 3D Spatial Benchmark MazeBench
Blogger patiencecave tested GPT-6 Astra on their self-built 3D open-world spatial reasoning benchmark MazeBench: the model ran for over 60 hours and scored just 14% without code execution, yet this is the best result on the benchmark to date—above GPT-5.6 Sol with Python (13%)—which the author reads as a generational capability leap, though far from passing MazeBench's harder challenges. Rohan Paul noted the score is roughly 7x that of comparison model Claude Fable 5.1. Reddit user WonderfulBuffalo32 posted a screenshot of GPT Astra scoring only 13% with no tools, arguing maze-type spatial reasoning lags far behind what official demos suggest.
Confirmed
- MazeBench is a long-horizon spatial reasoning benchmark: 100 gems hidden across 200+ rooms, requiring agents to rotate the view, manipulate boxes, recover from mistakes, and execute plans exceeding 100 steps; the no-Python track is harder since code-capable agents can solve levels in reverse.
- To save tokens, the eval allowed GPT-6 Astra to submit batched moves per turn; the model then consistently planned 10-20 steps ahead, cutting consumption from roughly 3 billion tokens to 350 million.
- Astra outperformed all prior agents, easily solving puzzles requiring 5 simultaneous moving parts (e.g., Klotski-style puzzles); exploration heatmaps show far broader coverage than Fable 5.1.
- Failure analysis shows Astra tends to interpret levels as top-down 2D planes, treating walls visible only from a 3D view as invisible barriers; it clears tutorial levels but fails 3D puzzles with more than 3 stacked objects.
Unconfirmed
- The Reddit post reports 13% for a no-tool run, slightly below patiencecave's 14%; whether the two evaluations used identical setups remains unverified.
Why it matters
Even the strongest new-generation models show structural weaknesses in long-horizon, true 3D spatial reasoning—"top-down map" world understanding remains a bottleneck. Meanwhile, batched planning's dramatic token reduction shows how much agent evaluation design affects cost.
2026-09-07 ~ 2026-09-08 · 8 related posts
- Episode 1: GPT-6 Astra Scores Just 14% After 60 Hours on 3D Spatial Benchmark MazeBench(2026-09-07, 8 posts)
- Episode 2: GPT-6 Astra on MazeBench: Bare Model Beats GPT-5.6 With Tools(2026-09-14, 6 posts)
Primary sources
- GPT-6 'Astra' burns 60+ hours in 3D spatial reasoning eval, scores just 14% — patience_cave ·
- GPT-6 without code execution beats GPT-5.6 with Python on MazeBench — patience_cave ·
- GPT Astra scores just 13% on MazeBench without tools — Wonderful_Buffalo_32 ·
- [source] GPT-6 'Astra' burns 60+ hours in 3D spatial reasoning eval, scores just 14% — patience_cave · 2026-09-07
- GPT-6 Astra outperforms all prior agents, juggling 5 moving puzzle parts — patience_cave · 2026-09-07
- GPT-6 treats 3D levels as top-down maps, failing puzzles stacked over 3 blocks — patience_cave · 2026-09-07
- Batched moves let GPT-6 plan 10-20 steps ahead, cutting 3B tokens to 350M — patience_cave · 2026-09-07
- [source] GPT-6 without code execution beats GPT-5.6 with Python on MazeBench — patience_cave · 2026-09-07
- [source] GPT Astra scores just 13% on MazeBench without tools — Wonderful_Buffalo_32 · 2026-09-07
- GPT-6 Astra scores 14% on no-Python MazeBench, 7x Claude Fable 5.1's 2% — rohanpaul_ai · 2026-09-08
1 near-duplicate retellings: basedjensen