Benchmark: Google's Gemini Flash Underperforms OpenAI's Luna in Bug Fixing and Cost
PawelHuryn · x · 2026-08-25
A benchmark test on real-world repositories shows that Google's cheapest agent model, Gemini 3.7 Flash, loses to OpenAI's GPT-5.6 Luna in both bug fixing performance and cost. Gemini 3.7 Flash fixed 22 out of 105 bugs in 96 minutes at $8.43, while GPT-5.6 Luna fixed 33 bugs in 86 minutes at just $1.80. The price gap is significant: Flash costs $3.75 per million output tokens, 3x more than Luna's $1.20, yet delivers inferior results.
Related event: GPT-5.6 Luna Beats Gemini 3.7 Flash in Bug-Fixing While Costing Far Less(3 posts)→
More from coding & agent
- AI software factories hinge on the Factory Loop, not faster code generation — Pavan_Belagatti · 2026-10-16
- 'Dropping a 3,000-line PR twenty minutes after ticket assignment' — AI coding meme goes viral — facontidavide · 2026-10-16
- Self Labs launches Agent Vault to let AI agents log in and pay with credit cards — EXM7777 · 2026-10-16
- Plane Agents: triggers, not prompts, are what turn chatbots into workflow agents — JosephJacks_ · 2026-10-16
- Claude eng lead Felix Rieseberg publishes tutorial on being more ambitious with AI — danshipper · 2026-10-16
- Qwen Code Desktop v0.25.2 ships managed-agent fixes and MCP config repair — github-actions[bot] · 2026-10-16