Artificial Analysis Intelligence Index v5 widens open-weight gap from 3 to 12 points
ResearchCrafty1804 · reddit · 2026-10-28
Artificial Analysis shipped a major v5 update to its Intelligence Index, and the community is pushing back.
- Open-weight gap widens: from 3 points to 12. Kimi K3.1 Pro (weights unreleased) tops open weights at 46; among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 — 26 points behind the leader and below Claude Haiku 5.5 (max, 36).
- Reproducibility concerns: 60% of v5 evaluations use private test sets or held-out answers, making independent verification impossible.
- Accuracy questioned: users flag counterintuitive placements like GPT-Terra-6.1 ranking above Fable-5.1.
The new index incorporates 9 evaluations including AA-Briefcase v1.1, Terminal-Bench-Science 0.1 and HLE-Diamond.
More from Models
- ChatGPT Pro usage limits cut in half in 3 days, devs voice frustration at OpenAI — johnlindquist · 2026-10-28
- Grok 4.7 scores just 24% on CritPt v2 physics benchmark, trailing Chinese models — teortaxesTex · 2026-10-28
- GPT-6.1 Terra ranks high on Frontier Code 2.0 at just $1 input pricing — bookwormengr · 2026-10-28
- Model builder confirms inheriting weights from Kimi with full retrain — basedjensen · 2026-10-28
- Steer just two neurons to change Qwen3-8B's voice: L8 N8749 +1, L9 N8774 -1 — dejavucoder · 2026-10-28
- ZCode offers 300M free GLM-5.3-Flash tokens daily, with a quirky claiming window — AccBalanced · 2026-10-28