Google engineer uses 1948 von Neumann quote to define evals vs. benchmarks
DynamicWebPaige · x · 2026-11-09
DynamicWebPaige, a Google engineer, shared a meme defining evals and benchmarks with a 1948 John von Neumann quote: "If you tell me precisely what it is a machine cannot do, then I can always make a machine which will do just that" — a wink at how benchmarks measure solved problems while evals probe the boundary of capability.
More from AGI Musings
- Takeoff debate: concave curves don't prove diminishing returns block superintelligence — austinc3301 · 2026-11-09
- Exponential AI progress is exponential investment with sublinear returns — world-changing, not god — dhadfieldmenell · 2026-11-09
- Takeoff debate: concave curves are by construction, not evidence of diminishing returns — austinc3301 · 2026-11-09
- Derya Tabak: People are still massively underestimating AI — abundant superintelligence changes everything — DeryaTR_ · 2026-11-09
- Counter to Sutton's "LLMs aren't bitter lesson enough": pretraining won via Hayekian aggregation — akbirthko · 2026-11-09
- fkasummer: 2028 hinges on a new paradigm for intelligence — transformers, backprop all up for grabs — fkasummer · 2026-11-09