SWE-bench creator quips it's time to invent 'unforced SWE-bench' as the benchmark saturates

OfirPress · x · 2026-11-21

SWE-bench creator and Princeton researcher Ofir Press jokes that with SWE-bench essentially saturated, the community now needs to figure out an "unforced SWE-bench" — a harder benchmark measuring genuine software engineering ability — and saturate that too. A wry take on how coding benchmarks can no longer distinguish top models.

Original post →

More from Fun

Fun channel →