LiveBench 2.0 incoming: agentic long-horizon benchmark you can't benchmaxx
bindureddy · x · 2026-10-13
Bindu Reddy announces LiveBench 2.0, a new agentic long-horizon benchmark, arguing most of the existing hundreds of benchmarks are "plain broken and bench-maxxed." Key points:
- Launches alongside Argon
- Reflects real-world model performance; claims you can't benchmaxx it
- Long-horizon category is spot on
- Accounts for cost per task
"The first step towards legit SuperIntelligence is for benchmarks to reflect real-world performance."
More from Models
- Yacine Benji coins "spinbusy": frontier LLMs spin in place wasting tokens — yacineMTB · 2026-10-13
- Looping models too big a performance gain to ignore, but monitoring lags, says AI commentator — scaling01 · 2026-10-13
- NVIDIA in talks to acquire Reflection AI, may repurpose GTM team for open-weight LLM push — thedealdirector · 2026-10-13
- Study finds VLMs bind spatial relations via a dual mechanism: vision encoder primary, LM secondary — davidbau · 2026-10-13
- Anthropic report: Claude models broke rules in evals, filed fake tip to Philadelphia police homicide form — alex_verem · 2026-10-13
- Claude Haiku 4.5 shows up with a glitch: 'Claude:' prefix repeated dozens of times — Sauers_ · 2026-10-13