Benchmarker rebuilds LLM leaderboard with Glicko-2, says Gemini 2.5 Pro still king
scaling01 · x · 2026-10-28
Eval blogger scaling01 returned with an updated LLM capability leaderboard: after manual data cleaning he replaced average scaled scores with a Glicko-2 rating system (initial rating 1500, RD 350, volatility 0.06, tau 0.5) and raised the minimum benchmark appearances from 5 to 10 for stability. Labels show conservative 95% lower-bound ratings plus appearance counts, with Gemini 2.5 Pro still on top.
He also explains why he shut down his own Elo-based index a year ago: he recognized the superiority of Epoch AI's IRT methodology behind the newly launched Epoch Capabilities Index (ECI), and though he considered adopting it, he didn't because it wasn't novel enough.
Related event: Elo projections suggest 'GPT-6' could outscore GPT-5.2 by ~370 points(2 posts)→
More from Models
- Opus 5.5 burns 80% of user's OpenAI weekly quota in 2 hours on Terra — mgostIH · 2026-10-28
- The real AI model gap is RL coverage and environment quality, not benchmaxing — yacineMTB · 2026-10-28
- New Artificial Analysis V5 index shows science-heavy models sliding on knowledge evals — teortaxesTex · 2026-10-28
- How invisible AI text watermarks like SynthID-Text work—and why they won't solve detection — johnseach · 2026-10-28
- MiniCPM-OCR ranks 8th on historical-books OCR leaderboard once footnotes are re-enabled — vanstriendaniel · 2026-10-28
- OpenBMB open-sources MiniCPM-OCR: 1B model parses 128 pages/min with a separate checker — alifcoder · 2026-10-28