Gemini 3.7 Flash loses to GPT-5.6 Luna in bug fix benchmark, costs 4.7x more
PawelHuryn · x · 2026-08-25
Benchmark results by Pawel Huryn reveal that Google's cheapest agent model, Gemini 3.7 Flash, underperforms compared to OpenAI's GPT-5.6 Luna. In a test of 105 planted bugs, Gemini 3.7 Flash fixed 22 bugs in 96 minutes at a cost of $8.43, while GPT-5.6 Luna fixed 33 bugs in 86 minutes for just $1.80. Gemini's performance also trailed behind Fable 5 and Grok 4.6, highlighting a significant price disadvantage.
Related event: GPT-5.6 Luna Beats Gemini 3.7 Flash in Bug-Fixing While Costing Far Less(3 posts)→
More from Models
- Unverified: internal snapshot labeled gpt-6-astra-api-priv-ev3 spotted on model tracker — lyraxana · 2026-10-16
- NVIDIA releases Nemotron 3.5 Bolt 74B MoE that fits in 24GB VRAM at 2-bit — NerdyRodent · 2026-10-16
- Muse Code 1.4.8 cuts token usage; all user quotas reset immediately — alexandr_wang · 2026-10-16
- Alibaba and Baidu to power key parts of Apple's AI services in China — nordicinst · 2026-10-16
- Cornell and Amazon revisit the 16x pretraining compute gap: diffusion LMs need only 3-8x — LucaAmb · 2026-10-16
- Image Decision Benchmark JevImageBench hits v0.3.0 with reshuffled Top 5 — airesearch12 · 2026-10-16