Artificial Analysis Intelligence Index v5: 60% now uses private test sets
ajratner · x · 2026-10-29
- Artificial Analysis released Intelligence Index v5, its biggest upgrade since January: 2 new benchmarks, 3 upgrades, and for the first time 60% of the Index is based on private test sets or held-out answers, mitigating benchmark contamination.
- New boards include FrontierCode 2.0 (using Cognition's newly launched benchmark) and Terminal-Bench-Science.
- It also launched the Progress Index v1, tracking model capability over time on one continuous scale for cross-generation comparison.
More from Models
- theo slams new model as worse than Composer 2 at 4x the cost, calling it Llama 4 tier — zacharynado · 2026-10-29
- Community speculates Gemini's new tier naming: Carbon, Argon, Flash — koltregaskes · 2026-10-29
- Kaggle kicks off Bench Summit, arguing today's AI benchmarks fail on relevance, compute and verification — MeganRisdal · 2026-10-29
- SWE-sweep appears in a Google DeepMind frontier model card within a month — OfirPress · 2026-10-29
- Dev Pushes Back on Google AI Hype: Great Benchmarks, Few Actual Users — zacharynado · 2026-10-29
- 'Gemini 3 was thoroughly benchmaxxed,' overtaken by Opus 4.5 within two weeks — zacharynado · 2026-10-29