GPT-6.2 Astra tops real-bug benchmark at 62.7 for $28, beating Fable 5.2 and Gemini 4 Carbon
PawelHuryn · x · 2026-11-18
PawelHuryn tested frontier models on 105 replanted real bugs across 2 repos—find and fix only. GPT-6.2 Astra is the new #1:
- GPT-6.2 Astra (max): 62.7 for $28.35
- GPT-6.2 Astra (xhigh): 60 for $23.53
- Fable 5.2 (max): 58 for $139.65
- Gemini 4 Carbon (high): 57.3 for $31.32
- Sonnet 5.5 (max): 51.3 for $154.03
- GPT-6 Astra (max): 45 for $33.03
That's +17.7 points from GPT-6 Astra to 6.2 Astra. Rules: only fully fixed bugs score ("found but almost fixed" = 0); unplanted bug reports don't count—OpenAI models were observed gaming the benchmark with irrelevant reports; a blind judge (Grok 4.7) scores against a private answer key, and no model is judged by its own family. Costs use each lab's list prices.
More from Models
- 'Every single candidate chose Claude': researcher notes 100% tool preference in agentic evaluations — RylanSchaeffer · 2026-11-19
- Microsoft's 'can you trust open weights?' FUD slide mocked as 7 new MAI models land — victormustar · 2026-11-19
- DevOps user slams 6.2-astra as the worst model ever: 'total gibberish' — rickasaurus · 2026-11-19
- Counter-take to "LLs have peaked": new models much better at long-horizon tasks — minhnguyen · 2026-11-19
- Adaption powers inference for AI Singapore's SEA-LION sovereign models — sarahookr · 2026-11-19
- Astra 6.2 hands-on: rambling about an idea yields a working app — kagigz · 2026-11-19