SWE-bench Co-author Ofir Press: Good Benchmarks Should Start Near 0% Accuracy

OfirPress · x · 2026-11-21

On Kaggle's Evaluated podcast, SWE-bench co-author Ofir Press outlined common mistakes benchmark builders make:

His design rule: tasks that are "easy to explain, easy to judge, but extremely hard to execute."

Original post →

More from Models

Models channel →