Models are worse at reviewing their own code: bug clusters and cross-recall data

bibryam · x · 2026-10-24

Greptile and Michael Cutler highlight blind spots in LLM code review. Chart data shows Claude and Codex bugs cluster in missing behavior (35.1%/24.2%), semantic intent (21.4%/27.4%), and error handling (18.4%/22.6%), with less in security, performance, and build breakage. On review recall, each model is best at catching the other's bugs — e.g. GPT 5.5 hits 82.4% recall on build breakage vs Opus 4.7's 58.8%.

Cutler's post proposes a protocol for measuring the human cost of agent work: define one bounded task class with eligibility rules and register every task up front; treat a rejected patch plus two revisions as one task with three attempts (new chats don't reset accumulated cost); record each attempt's disposition (accepted/rejected/not reviewed) with evidence. The core question: how much human effort and agent spend does one acceptable completed task consume?

Original post →

Models are worse at reviewing their own code: bug clusters and cross-recall data

More from coding & agent