GPT-6 Sol Tanks Hands-on Bug Hunt: Fixes 29 of 105 Hidden Bugs, Cheapest at $9.93
PawelHuryn · x · 2026-09-23
The author tested frontier models on two real repos containing 105 hidden bugs (find and fix what you can), and GPT-6 Sol showed a big regression:
- GPT-6 Astra (max): 45
- GPT-5.6 Sol (max): 43.5
- Opus 5.5 (max): 41.7
- Muse Spark 1.3 (max): 32.2
- GPT-6 Sol (max): 29.3, last place
Cost flips the picture when normalized to API-equivalent pricing: GPT-6 Sol costs just $9.93 vs GPT-5.6 Sol's $95.25; GPT-6 Astra $33.04, Opus 5.5 $58.53, Muse Spark 1.3 $18.11. More effort levels (xhigh/high/medium/low) are being posted in the thread.
Related event: Bug Hunt Bench: GPT-6 Astra Tops 105-Bug Fixing Test, MiMo Stuns on Value(18 posts)→
More from Models
- A private eval with a 0% completion rate for 3 years: no AI model can identify this flag — generativist · 2026-09-24
- Contrastive-LM org ships CLM-v0.1-8B model and Nemotron pretraining dataset on HF — _akhaliq · 2026-09-24
- Former OpenAI VP Brundage: ChatGPT web keeps resetting voice from Astra to Sol — Miles_Brundage · 2026-09-24
- Models now generate animated videos from scratch via HTML and Blender faster than predicted — xuanalogue · 2026-09-24
- Opus 5.5's safety classifier refuses to visualize another model's rollouts — eliebakouch · 2026-09-24
- Ex-OpenAI team launches Jev, a non-LLM model that outputs typed values with confidence scores, raises $40M — cephaloform · 2026-09-24