Artificial Analysis v5: open-weight models fall 12 points behind, gap tripled from v4.3
steipete · x · 2026-10-28
Artificial Analysis moved to closed benchmarks in v5 (retiring GDPval-AA) and found open-weight models falling further behind than previously measured:
- Best scorer Kimi K3.1 Pro (weights not yet released) hits 46, 12 points behind the leader, vs. a 3-point gap in v4.3
- Among models with released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) top out at 32; Claude Haiku 5.5 (max) outscores them all
- Gaps are widest on HLE-Diamond and Terminal-Bench-Science: Kimi K3.1 Pro scores 37%/34% vs. 68% for Claude Opus 5.5 (max) and 63% for GPT-6 Astra (max)
Ethan Mollick urges caution on the data but notes it matches the vibe that no Fable-class open model has shipped recently.
More from Models
- New Paper Shows Gemini-Found Softmax Bug Fix Still Fails Property Tests — brandondamos · 2026-10-28
- Reddit users complain ChatGPT's coding quality keeps getting worse — LSP-86 · 2026-10-28
- Cryptic insider tease sparks guesses: GPT-6.2 Sol or GPT-6.1 Astra? — CtrlAltDwayne · 2026-10-28
- ECI Metric Adopted by Anthropic, Google and Artificial Analysis — Jsevillamol · 2026-10-28
- 'The OpenAI Canyon' meme chart shows OpenAI models trailing on output tokens — steipete · 2026-10-28
- First open-weight production dLLM: Clover-35B hits 680 tps on one GPU, beats same-size AR models on SWE-bench — taoyds · 2026-10-28