Gemini 4 Argon fixes just 33 of 105 real bugs, trailing GPT-6 Astra's 45 in hands-on bench
PawelHuryn · x · 2026-10-14
Pawel Huryn tested Gemini 4 Argon on his Bug Hunt Bench — two real repos with 105 planted bugs that frontier models missed in Q1 2026:
- GPT-6 Astra (max): 45
- Fable 5.1 (max): 43
- Opus 5.5 (max): 41.7
- Gemini 4 Argon (high, its max tier): 33
- Muse Spark 1.3 (max): 32.2
- Grok 4.7 (xhigh): 29
Verdict: a big step up from Gemini 3.8 Flash, but still 10 points behind the OpenAI and Anthropic flagships; historical uncertainty is 2-3 points. The full leaderboard also shows Sonnet 5.5 (max) on top at 51.3/105, at the cost of 287 minutes and $154 per run.
More from Models
- Apollo eval finds Gemini 4 Argon treats its chain-of-thought as audience-facing — nitarshan · 2026-10-14
- One prompt, three personalities: GPT, Gemini and Opus trade sparkling jokes — emollick · 2026-10-14
- Gemini Argon's table preference plunges to 31.5% from Flash's 85.9% in new test — dejanseo · 2026-10-14
- User blasts Claude for refusing to click a Terms & Conditions button — Angaisb_ · 2026-10-14
- Sonnet 5.5 tops Coding Agent Index; xhigh delivers 93% of the score for 23% of the cost — ArtificialAnlys · 2026-10-14
- Snap launches as first fully sub-quadratic LLM in production with constant compute per token — ryanallred · 2026-10-14