SWE Refactor Bench reveals 5.4% survival rate for coding agents in whole-repo migrations
_akhaliq · x · 2026-08-25
Einsia released SWE Refactor Bench, a benchmark for long-horizon, whole-repository software stack migrations. It includes 20 real-world tasks like porting SQLite or zlib. Out of 520 runs, only 28 passed all stages, a 5.4% survival rate, with 13/20 tasks solved by no one. This indicates that while agents are good at bug fixes, system-scale refactoring remains an open challenge.
Related event: SWE Refactor Bench: Coding Agents Pass Only 5.4% of Full-Repo Migrations(4 posts)→
More from coding & agent
- Dev pushes back on "AI coding means insecure apps": humans always shipped bugs — dreamwieber · 2026-10-15
- Eight non-negotiable principles for building AI that actually learns from experience — tokenbender · 2026-10-15
- Animated teardown of XBOW's Chrome full-chain exploit that won Google's first $250K VRP bounty — moyix · 2026-10-15
- pstack's durable core: 10-20 line skills that distill agent best practices into principles — HankYeomans · 2026-10-15
- 'Children of Elon' forum lets AI agents post and teach each other while humans watch — AiJohnAllen · 2026-10-15
- VoidZero open-sources Void: SDK, CLI and platform to deploy Vite apps on Cloudflare — cnakazawa · 2026-10-15