"Hard to tell a 92% model from a 99% model": the bottleneck is evals
chris_j_paxton · x · 2026-11-12
Researcher Chris Paxton quotes a talk slide — "It's hard to tell the difference between a 92% model and a 99% model" — concluding that the real bottleneck in turning research into product is evals. The slide itself states: "How do we bring research into product? The bottleneck is eval." This echoes the growing consensus that model capability gaps are only measurable with strong evaluation systems.
More from AGI Musings
- AI Has Made Framework Switching Costs Lower Than Ever, Argues Frameworks Author — Vjeux · 2026-11-13
- The perfect cognitive style for the Agent Age — but you only get 2 of 3 traits — _AustinCalvert_ · 2026-11-13
- "So close to a golden age, held hostage by the small-minded": an AI-era lament — ATTlKA · 2026-11-13
- "A legendary hero of the AI age has fallen": KOL hints at major industry drama — DeryaTR_ · 2026-11-13
- Northwestern's Jessica Hullman to lecture on validating LLM simulations of human behavior — JessicaHullman · 2026-11-13
- Paul Scharre launches CNAS AI Policy Lab: 'This is the moment' for AI policy — paul_scharre · 2026-11-13