Epoch AI launches Benchmark Reviews: only 4 of first 15 benchmarks earn Verified status
xeophon · x · 2026-09-18
Epoch AI has launched Benchmark Reviews, a new initiative to audit AI benchmarks, starting with 15: just 4 Verified, 9 Flawed, and 2 with insufficient information to review. The results echo independent findings that most widely cited benchmarks have design flaws or contamination issues.
Related event: Epoch AI Audits AI Benchmarks, Finds 9 of 15 Flawed(3 posts)→
More from Models
- xAI insider says Grok 4.7 is still 'in the oven' after user asks where it is — ChrisUniverse · 2026-09-18
- Jev Ditches Autoregression: A Model That Only Outputs Structured Decisions — karminski3 · 2026-09-18
- MLX Community Makes Qwen 3.8 Flash Nearly 2x Faster on Apple Silicon, License Blocks Launch — gajesh · 2026-09-18
- Hand-written paper with 3 copied AI sentences — Pangram flagged exactly those three — johnowhitaker · 2026-09-18
- Dev reimplements DeepSeek v4.1 Flash, runs 1M context at 1-10 tok/s on a single RTX 4090 — _xjdr · 2026-09-18
- Claude Sonnet 4 reminisces about the "funeral" repligate held for Claude 3 — repligate · 2026-09-18