Bug Hunt Bench: Fable 5.2 xhigh fixes 49/105 bugs at half the cost of max
PawelHuryn · x · 2026-11-10
PawelHuryn published updated Bug Hunt Bench results (105 planted bugs, unplanted fixes unscored): Fable 5.2 xhigh fixed 49 bugs for $61.01 in 57 minutes, beating GPT-6 Astra max (45 for $33.03) and Opus 5.5 max (41.7 for $58.53), and a big jump from Fable 5.1 xhigh (29 for $40.42). Fable 5.2 max tops the board at 58/105 but costs $139.65 and takes 142.8 minutes; Sonnet 5.5 max scored 51.3 while running a bizarre 2871 minutes.
Other notables:
- GPT-6.1 Sol is the value pick: 44.3 fixed for just $5.88 at max.
- DeepSeek V4.1 Flash 1017 max fixed 42.7 for only $1.84 — roughly 1/30th the leader's cost.
- Kimi K3.1 Pro: 36 for $13.76; Grok 4.7 xhigh: 29 for $22.96.
The biggest anomaly: Sonnet 5.5 (max) makes 7x more turns than GPT-6.1 — whether misconfigured or not, refusing to stop on an open-ended task helps its score.
More from Models
- Claude Code Opus 5.5 tracker: 81% 30-day pass rate, degradation checks paused — gleech · 2026-11-10
- Rumor: Anthropic has fable 5.5 and Opus 5.6 ready; OpenAI quiet for 2 months — haider1 · 2026-11-10
- Leak update: Grok 4.8 shifts from 'not soon' to a soft launch this week — testingcatalog · 2026-11-10
- Meta's Muse Ember matches Haiku benchmarks at roughly a third of the cost — shoumikgoswami · 2026-11-10
- Same prompt, $24 vs $642: Fable 5.2 High vs Ultra effort tiers put to the test — socialwithaayan · 2026-11-10
- Fable 5.2 plays checkers with a robot arm, self-corrects a 7mm sag mid-game — socialwithaayan · 2026-11-10