Artificial Analysis Launches Intelligence Index v4.2 With 40% Private Test Sets to Block Benchmark Gaming
ArtificialAnlys · x · 2026-09-05
Artificial Analysis shipped an interim v4.2 of its Intelligence Index, pulling forward pieces of v5 eight months after v4 debuted, citing the rapid pace of frontier model launches.
- New AA-Briefcase: an in-house agentic knowledge-work eval with a private held-out test set; models complete multi-week projects with thousands of source files, graded via rubric and pairwise comparison on task success, analytical quality, and presentation.
- New GDP.pdf (by @HelloSurgeAI): single-turn professional document reasoning across 100 PDFs and ten domains, synthesizing evidence from 4,592 pages and scored against 1,275 expert-authored criteria with an all-pass headline metric.
- Anti-gaming: private held-out weight doubled to 40% of the Index (AA-Briefcase, AA-Omniscience, CritPt solutions), set to rise further in v5.
- Saturated GPQA Diamond removed; grading infrastructure, sampling, and Elo re-anchoring improved.
More incremental releases are planned as v5 development continues.
More from Models
- Ask your AI agent its favorite color — it apparently has one — every · 2026-10-21
- Google demos Gemini 4 Argon building a 3D periodic table synced to Tom Lehrer's 'The Elements' — sundarpichai · 2026-10-21
- MiniMax follow-up: M3.1 Flash teased after GTC release wave — MaziyarPanahi · 2026-10-21
- Claude Max 5x limits feel generous: heavy user can't burn through weekly quota — Angaisb_ · 2026-10-21
- General Intuition announces API coming soon, pitching itself as the frontier lab for acting in space and time — chris_j_paxton · 2026-10-21
- Stronger Model, Weaker Product: Dev Defaults to Fable 6.1 Sol Ultra Over 5.1 — curious_vii · 2026-10-21