Artificial Analysis v5 goes 60% private: open models look far worse when benchmarks stop leaking
rickasaurus · x · 2026-10-29
Artificial Analysis released Intelligence Index v5, its biggest upgrade since January: for the first time 60% of the composite index is built on private test sets or held-out answers, with new benchmarks like FrontierCode 2.0 and a new Progress Index tracking capability over time on one continuous scale.
Quoting the news, LLMJunky highlights a telling pattern: when benchmarks stop being public, open models score significantly worse. He rates DeepSeek 4.1 Flash and GLM 5.3 as genuinely impressive, but pushes back on the "China caught up" / "Opus at home" narrative, arguing the industry now ignores nearly every public benchmark and hopes private evaluation restores credibility.
More from Models
- ChatGPT web users report merged usage limits draining quota across all surfaces — teortaxesTex · 2026-10-29
- Anthropic says no regression behind Claude Code usage complaints, blames heavy sessions — bcherny · 2026-10-29
- OpenAI: Codex and Chat capabilities will merge this year, unified limits feared — CtrlAltDwayne · 2026-10-29
- Community reverse-engineers Qwen3.5-27B from public weights: it's grown from the 9B model — davidad · 2026-10-29
- Reddit User: AI Labs' Tier Sprawl Is Broken — One Model, One Price Would Win — Tight-Grocery9053 · 2026-10-29
- Mystery model 'Kerria' spotted on LMArena, tokenizer errors suggest it could be Qwen's next model — lyraxana · 2026-10-29