Harvard paper: Higher coding benchmark scores didn't translate into more real work
A Harvard paper (arXiv:2611.23216) finds that coding models' benchmark scores rose by 11 points while their real-world code merge rates showed no improvement, fueling debate over whether benchmarks reflect actual productivity.
2026-11-23 ~ 2026-11-23 · 2 related posts
- Harvard paper: higher SWE-Bench scores didn't translate into more merged pull requests — rohanpaul_ai · 2026-11-23
1 near-duplicate retellings: rohanpaul_ai