LiveBench easily gamed; 2.0 coming to fix it
bindureddy · x · 2026-08-23
Bindu Reddy pointed out that agentic coding on LiveBench is easily "benchmaxxed" (gamed), casting doubt on current rankings like Qwen 27B beating GPT 5.6. The team is working on LiveBench 2.0, which will be harder to exploit.
Related event: LiveBench Found Vulnerable to Benchmark Gaming, 2.0 in Development(2 posts)→
More from Models
- Meta releases Muse Spark Open: 816B open-weight model that runs on two M5 Ultra Mac Studios — op7418 · 2026-10-16
- Meta open-sources Muse Spark Open: 816B MoE that runs on two Mac Studios — op7418 · 2026-10-16
- Robotics LLMs tested on 16 tasks: GPT-6 Astra hits 17.2% success vs Fable's 9.1% — pabbeel · 2026-10-16
- Meta open-sources Muse Spark: top US open model, scores just below DeepSeek — bindureddy · 2026-10-16
- TRL adds day-0 support for fine-tuning Muse Spark Open's 816B MoE with LoRA on 64 H100s — QGallouedec · 2026-10-16
- Muse Spark Open eval configs revealed: inference parameters for GLM-5.3, Kimi K3, Qwen comparisons — evijit · 2026-10-16