Proactive Bug-Finding Agents Report Under 20% of Bugs Humans Fix, but Find Many Other Valid Bugs
jyangballin · x · 2026-10-28
New paper "Frontier Coding Agents Find Different Bugs Than Maintainers Fix" studies proactive bug finding:
- Agents report fewer than 20% of the bugs humans fixed
- But they also report many other bugs, most of which are valid — pointing to an important distributional mismatch
jyangballin frames this as a promising benchmark direction: SWE-sweep, SWEeper-bench, and now PROBE mostly make SWE-bench-style tasks harder for difficulty's sake, while proactivity asks models to take on meaningful labor we're still doing ourselves, like writing issues.
Related event: Coding Agents Find Different Bugs Than Maintainers Fix, Paper Shows(3 posts)→
More from coding & agent
- Why parallel Neo4j imports run slower than serial: lock contention and the fix — JeremyCMorgan · 2026-10-29
- Generating K-Pop music videos with a Claude Music Video skill, no MCP needed — AIandDesign · 2026-10-29
- Pydantic's code mode cuts agent tokens 30% and speeds long tasks 51% across 8 models — samuelcolvin · 2026-10-29
- Decagon details a self-improving autoresearch loop for inference stack optimization — kimberlywtan · 2026-10-29
- Dev uses Claude-built library to ship Obsidian plugin that plans music practice with spaced review — dSebastien · 2026-10-29
- Sitzmann open-sources texwerk, a self-hosted collaborative LaTeX editor with native agent support — vincesitzmann · 2026-10-29