Artificial Analysis Intelligence Index v5: 60% private datasets, open-weights gap widens to 12 points
ArtificialAnlys · x · 2026-10-28
Artificial Analysis launched Intelligence Index v5, its biggest upgrade since v4 in January: 2 new benchmarks, 3 upgrades, and 60% of the Index now built on private test sets or held-out answers to resist contamination.
New benchmarks
- FrontierCode 2.0 (from Cognition): 132 real open-source repo tasks graded on whether a maintainer would merge the agent's change
- Terminal-Bench-Science: 70 agentic scientific research tasks run in a terminal
Key results
- Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55); Gemini 4 Argon (high) follows at 50
- The open-weights gap widens from 3 to 12 points: Kimi K3.1 Pro (weights unreleased) tops at 46; among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32, below Claude Haiku 5.5 (max, 36)
- Cost per task varies 13x+ among leaders: GPT-6.1 Terra (max) cheapest at $0.96, Claude Fable 5.1 (max) priciest at $13.04
A companion Progress Index v1 tracks capability over time on one continuous scale.
More from Models
- Debate: Claude isn't 'lobotomized' — inference stack updates may shift old-model outputs — JasonBotterill · 2026-10-28
- AI labs near IPO, yet LLMs still produce 'astonishingly terrible prose' by default — GabGarrett · 2026-10-28
- Rebuttal: Kimi 3.1 Nears Fable on Frontier Code 2, Contradicting 'China Models Crushed' Claim — bookwormengr · 2026-10-28
- Open model Flint hits #2 trending on Hugging Face, team outlines next steps — josh_tobin_ · 2026-10-28
- Artificial Analysis Intelligence Index v5 widens open-weight gap from 3 to 12 points — ResearchCrafty1804 · 2026-10-28
- YacineMTB rails: even the best LLMs are 'complete dogshit' and he's tired of it — yacineMTB · 2026-10-28