AgentBug-Smith paper: top coding agents fix just 9% of agent harness bugs
rohanpaul_ai · x · 2026-10-03
The author shares the arXiv link for AgentBug-Smith: Automatically Reproducing Real-World Harness Bugs in Agentic Systems, by researchers from UChicago, Fudan, TensorBlock, Tsinghua and UIUC.
The method automatically reproduces real-world agent harness bugs, outperforming existing general-software bug reproduction techniques by 10.67%–27.56% across backbone LLMs. It yields Live-Harness-Bench, a live benchmark of 200 reproducible harness bugs. Systematic evaluation shows state-of-the-art software agents struggle to repair real harness bugs, while distilling lessons from past fixes significantly improves repair performance.
More from coding & agent
- New TLAPS-Bench blog: formally proving correctness of real-world system specs with TLA+ — tianyin_xu · 2026-11-19
- Rivet's lesson: JEV doesn't reason—split severity classification into tiny yes/no prompts — yawnxyz · 2026-11-19
- 16x Microsoft MVP demos a native PowerShell agent built on Microsoft's Intelligent Terminal — dfinke · 2026-11-19
- Musk: Treat Grok Bot like a really smart employee, says most users underuse it — elonmusk · 2026-11-19
- Eve agent tool cuts build time 8x to 6.9s by reusing compiler module graph — evilrabbit_ · 2026-11-19
- Dev shares NL-driven testing UI: 'the best interface is chat' — KlausCodes · 2026-11-19