Opus 5.5 fixes 43 of 105 planted bugs, nearly matching GPT-6 Astra at double the cost
PawelHuryn · x · 2026-09-23
The author planted 105 bugs across two real repos and benchmarked flagship models (max tier) on finding and fixing them, tracking cost:
- GPT-6 Astra (max): 45 fixed for $33.03 (n=3)
- Opus 5.5 (max): 43 fixed for $60.49
- Fable 5.1 (max): 43 fixed for $77.55
- Muse Spark 1.3 (max): 32.2 fixed for $18.11
- Opus 5.0 (max): 27 fixed for $51.33
Takeaway: Opus 5.5 essentially closes the gap with GPT-6 Astra and Fable 5.1, a big jump over Opus 5.0 (27), though at a higher cost per fix. More effort levels and attempt counts to follow in the thread.
Related event: Bug Hunt Bench: GPT-6 Astra Tops 105-Bug Fixing Test, MiMo Stuns on Value(18 posts)→
More from coding & agent
- As models improve, writing good evals gets harder and more important — amankhan · 2026-11-12
- Omarchy 4.6 lets you share local AI models across your tailnet, says DHH — alexcovo_eth · 2026-11-12
- Coding harnesses hand shell access to inference providers — 4-step gateway mitigation — shensi · 2026-11-12
- Slack's David Hoang: agents beat people on coordination, proposes 'boundary agents' — davidhoang · 2026-11-12
- CS student mourns coding flow states he'll never experience in the AI era — chrissyykat · 2026-11-12
- AI Engineer Code Summit features its first all-female keynote lineup — altryne · 2026-11-12