Artificial Analysis Intelligence Index v4.2 drops saturated GPQA, adds private test sets to curb gaming
SonglinYang4 · x · 2026-09-06
Artificial Analysis shipped Intelligence Index v4.2, accelerating parts of its v5 release to keep pace with the frontier. cHHillee highlighted the meta-point: labs can benchmaxx, but benchmarks themselves can "benchmaxx-maxx" by redesigning to look better.
Key changes in v4.2:
- New AA-Briefcase: an agentic knowledge-work eval with a private test set
- New GDP.pdf (from HelloSurgeAI): long-context document reasoning across 4,592 PDF pages
- Removed GPQA Diamond: the scientific reasoning benchmark is now saturated
- Higher weighting on held-out private test sets to prevent gaming, plus grading infrastructure upgrades for robustness
The direction: harder, more realistic tasks with less optimizable public test surface.
More from Models
- Ask your AI agent its favorite color — it apparently has one — every · 2026-10-21
- MiniMax follow-up: M3.1 Flash teased after GTC release wave — MaziyarPanahi · 2026-10-21
- Claude Max 5x limits feel generous: heavy user can't burn through weekly quota — Angaisb_ · 2026-10-21
- General Intuition announces API coming soon, pitching itself as the frontier lab for acting in space and time — chris_j_paxton · 2026-10-21
- Stronger Model, Weaker Product: Dev Defaults to Fable 6.1 Sol Ultra Over 5.1 — curious_vii · 2026-10-21
- User claims switching from Claude Opus 5.5 to GPT-6.1 dramatically slowed his machine — koltregaskes · 2026-10-21