56 AI practitioners vote on which benchmarks best indicate a frontier LLM: Terminal-Bench 4 tops with 25 votes
eliebakouch · x · 2026-11-16
Chris Barber asked 56 people in AI to pick, from 63 benchmarks, which ones they'd most trust to indicate whether an LLM is frontier. Results: Terminal-Bench 4 led with 25 votes, followed by Terminal-Bench Science (20), FrontierCode (13), FrontierSWE (10), Humanity's Last Exam (8), and Vending-Bench (7). AutomationBench, DeepSWE, and PostTrainBench got 5 each. The most common comment: people don't really trust existing benchmarks. Rules barred voting for benchmarks one's own company worked on, indexes, or human-voted boards. Full list at frontierlist.com/top-benchmarks.
More from Research
- Chollet: reasoning chain generation is search, not interpolation — teortaxesTex · 2026-11-16
- Factorised output heads let tiny LMs use a 131k vocab at equal training FLOPs — BLCNYY · 2026-11-16
- Terence Tao: math has moved from proof scarcity to proof abundance — haider1 · 2026-11-16
- Wrong AI advice hurts doctors' judgment more than right advice helps: study of 16,641 decisions — ValerioCapraro · 2026-11-16
- Free online book 'The Science of Computing' links PDEs, numerics, and cache performance — burkov · 2026-11-16
- Controlled speech experiments: joint training from scratch always beats separate pretraining — giffmana · 2026-11-16