50-PR Benchmark: GPT-6 Astra Finds 92 Bugs, GPT-5.6 Luna Matches 75% at 3.6% of the Cost
entelligenceai17 · reddit · 2026-09-14
Entelligence benchmarked GPT-5.6 Luna vs GPT-6 Astra on 50 real PRs from Cal, Sentry, Discourse, Keycloak, and Grafana:
- Astra found 92 confirmed bugs vs 69 for Luna
- Yet Luna caught 75% of the bugs at just 3.6% of the cost
- Full eval breakdown includes cost, avg output tokens, latency, precision, and bug classes like data/logic, security, and concurrency
- Next up: an Astra vs Fable 5.1 benchmark, with feedback requested on the evaluation
Takeaway: flagships win on recall, but cheaper models are highly cost-effective for code review.
More from coding & agent
- Creator builds shipwreck diving game with Opus 5.5 and Astrocade MCP — _amirabs · 2026-10-31
- GitHub Universe talk: 'Build once, run on any agent' with slides and skill files — 0xkarasy · 2026-10-31
- Claude can build pro software for ~$20k in tokens — and TAM shrinks to match — bigblueboo · 2026-10-31
- Microsoft launches Copilot harness in Foundry Agent Service for managed agent loops — WirelessLife · 2026-10-31
- Codex Remote live activities go live in the ChatGPT iOS app for tracking agent tasks — SIGKITTEN · 2026-10-31
- Open-source pixel FX bring a 3D wizard battle to life with slimes, archers and a golem — majidmanzarpour · 2026-10-31