OpenAI's cheapest model beats Google's in coding bug fix benchmark
LyalinDotCom · x · 2026-08-25
OpenAI's cheapest agent model, GPT-5.6 Luna, outperformed Google's cheapest, Gemini 3.7 Flash, in a bug fix benchmark. Luna fixed 33 of 105 bugs in 86 minutes for $1.80, while Gemini fixed only 22 in 96 minutes for $8.43. The test used blind judges on real repos.
Related event: GPT-5.6 Luna Beats Gemini 3.7 Flash in Bug-Fixing While Costing Far Less(3 posts)→
More from coding & agent
- Are the fastest GPU kernels now written by AI? Kernels are hyper-verifiable, argues one dev — fleetwood___ · 2026-10-16
- Anthropic lead engineer's 4-question prompt makes agents more ambitious, now a free skill — daniel_mac8 · 2026-10-16
- ChainForge dev shows 'feature map' for orchestrating subagents on large PRs — IanArawjo · 2026-10-16
- aiDotEngineer benchmarks agent orchestration pipelines, staging humans as Foreman and workers — vikvang1 · 2026-10-16
- Developer mass-produces 200 videos in hours using Claude Code + Hyperframes — mcraddock · 2026-10-16
- Penny University hosts software factory discussion; dev shares $40/mo vendor-agnostic AI stack — JnBrymn · 2026-10-16