SWE Refactor Bench: Coding Agents Pass Only 5.4% of Repo-Wide Migrations

The new SWE Refactor Bench tests coding agents on 20 real repo-wide stack migration tasks; across 520 runs only 5.4% passed, with the best model Opus 5 scoring just 47, exposing major gaps in long-horizon refactoring.

2026-08-25 ~ 2026-08-26 · 2 related posts

• SWE Refactor Bench reveals 5.4% survival rate for coding agents in whole-repo migrations — _akhaliq · 2026-08-25
• SWE Refactor Bench: Only 5.4% of 520 Runs Pass, Best Model Opus 5 Scores 47/100 — YouJiacheng · 2026-08-26