GPT-6-Astra tops GameDevBench amid benchmark doubts

GPT-6-Astra topped GameDevBench with surprisingly low scores, prompting Greg Kamradt to inspect the benchmark; he found tasks average 4.7 files and 114 lines of changes, making it more of an engineering test than game development.

2026-09-09 ~ 2026-09-10 · 3 related posts

• GPT-6-Astra tops GameDevBench, yet the raw score shocks: game dev far from solved — sethkarten · 2026-09-09
• Greg Kamradt questions GameDevBench: avg task edits 4.7 files — 'seems like engineering, not games' — GregKamradt · 2026-09-10
• Greg Kamradt digs into gamedevbench: reference solutions edit 4.7 files, 114 lines on average — GregKamradt · 2026-09-10