Ofir Press: the human baseline era is over — new guidelines for building benchmarks

OfirPress · x · 2026-10-13

Researcher Ofir Press argues that in the human-level AI era we could translate human tasks into benchmarks, always reporting three numbers: what the task is, top-model solve %, and human solve %. But in many domains — especially coding tasks — that era is over: tasks are now so hard that a human given reasonable time would score 0% on these coding benchmarks.

He calls this the end of the human-baseline era in some subdomains and has published guidelines for how to build benchmarks in this new era.

Original post →

Ofir Press: the human baseline era is over — new guidelines for building benchmarks

More from Research