Harvard paper: Higher coding benchmark scores didn't translate into more real work

A Harvard paper (arXiv:2611.23216) finds that coding models' benchmark scores rose by 11 points while their real-world code merge rates showed no improvement, fueling debate over whether benchmarks reflect actual productivity.

2026-11-23 ~ 2026-11-23 · 2 related posts

1 near-duplicate retellings: rohanpaul_ai