SWE-bench creator quips it's time to invent 'unforced SWE-bench' as the benchmark saturates
OfirPress · x · 2026-11-21
SWE-bench creator and Princeton researcher Ofir Press jokes that with SWE-bench essentially saturated, the community now needs to figure out an "unforced SWE-bench" — a harder benchmark measuring genuine software engineering ability — and saturate that too. A wry take on how coding benchmarks can no longer distinguish top models.
More from Fun
- Meme: asking Claude to prove the Riemann hypothesis because the stock is tanking — rachittshah · 2026-11-21
- 'We taught sand to think, then showed it its own reflection in a mirror of sand' — signulll · 2026-11-21
- Claude's 'Riemann solved' timeline called museum-worthy in viral AI moment — Kyrannio · 2026-11-21
- Meme: AI 'solves the Riemann hypothesis' then concludes building is bearish — adamamcbride · 2026-11-21
- "I have a few tokens to spare": user jokes about AI solving Millennium Problems — arjunrajlab · 2026-11-21
- Apple security hands out rainbow-themed swag boxes to researchers at OBTS — whurley · 2026-11-21