105 Real Bugs Tested: GPT-6.1 Sol Over 10x Cheaper Than GPT-5.6 Sol

PawelHuryn · x · 2026-10-23

The author ran his own Bug Hunt Benchmark (2 repos, 105 bugs that frontier models missed in early 2026) on Opus 5.5, Sonnet 5.5, and GPT-6.1 Sol, and calculated how much API value each subscription plan (Claude Max / ChatGPT Pro / SuperGrok) really buys — 9.3x to 137x its price.

Key findings:

Opus 5.5 is significantly stronger than Opus 5 and nearly matches Fable 5.1 (41.7 vs 43) at two-thirds of the cost ($58.53 vs $87.18); unless budget is unlimited, switch off Fable 5.1.
Sonnet 5.5 costs half of Opus 5.5 on input/output tokens ($2/$10 vs $4/$20 per MTok), but in long agentic sessions most cost is cache reads, priced identically at $0.20/MTok. Sonnet 5.5 (max) surprisingly topped the benchmark (51.3/105) by running 1,497 turns.
GPT-6.1 Sol is slightly stronger on complex tasks yet over 10x cheaper than GPT-5.6 Sol: half-price input/output and 4x cheaper cache reads (framed as a 95% discount by OpenAI, possibly temporary, likely returning to the standard 90%). The 10x gap is mostly explained by turns (485 vs 200 on the same task).
On launch day GPT-6.1 Sol ran nearly 2x slower than Opus 5.5; OpenAI's Tibo said it would get almost 2x faster within hours.

Original post →

105 Real Bugs Tested: GPT-6.1 Sol Over 10x Cheaper Than GPT-5.6 Sol

More from coding & agent