105 Planted Bugs Put Grok 4.7 at 28.7 vs GPT-6 Astra's 45 in Real-Repo Coding Test
PawelHuryn · x · 2026-09-22
Pawel Huryn ran a hands-on eval: two real repos, 105 planted bugs, find-and-fix, three runs per model. Results: GPT-6 Astra (max) 45, Muse Spark 1.3 (max) 32.2, Grok 4.7 (xhigh) 28.7, Grok 4.6 (xhigh) 28.7 — identical, not a typo — Opus 5 (max) 27, Qwen3.8-Max (max) 25.7. His verdict: "that's not the model we've been waiting for," as Grok 4.7 shows no gain over 4.6. Higher-effort scores and n=5 runs are being posted progressively in the thread to break the tie.
Related event: Bug Hunt Bench: GPT-6 Astra Tops 105-Bug Fixing Test, MiMo Stuns on Value(18 posts)→
More from Models
- Opus 5.5 feels nerfed — user blames system prompt restrictions, not the model — oyacaro · 2026-11-12
- ChatGPT now links to brand homepages far more, reshaping brand-mention tracking — lilyraynyc · 2026-11-12
- YC-backed Monte launches post-training platform to build proprietary models from your agents' real work — ycombinator · 2026-11-12
- Leak Claims OpenAI Will Launch Entire GPT-6.2 Family Tomorrow to Counter Opus 5.5 and Fable 5.2 — heypearlai · 2026-11-12
- ARC-AGI-3 leaderboard: models score 90%+ but at $10K+ per task — haider1 · 2026-11-12
- LMArena Launches Agent Arena for Head-to-Head Agent Evaluation — arena · 2026-11-12