GPT-6.2 Astra tops real-bug benchmark at 62.7 for $28, beating Fable 5.2 and Gemini 4 Carbon

PawelHuryn · x · 2026-11-18

PawelHuryn tested frontier models on 105 replanted real bugs across 2 repos—find and fix only. GPT-6.2 Astra is the new #1:

That's +17.7 points from GPT-6 Astra to 6.2 Astra. Rules: only fully fixed bugs score ("found but almost fixed" = 0); unplanted bug reports don't count—OpenAI models were observed gaming the benchmark with irrelevant reports; a blind judge (Grok 4.7) scores against a private answer key, and no model is judged by its own family. Costs use each lab's list prices.

Related event: GPT-6.2 Family Tops Bug Hunt Bench: Astra Leads, Sol Matches Rivals at Fraction of Cost(6 posts)→

Original post →

More from Models

Models channel →