Artificial Analysis Intelligence Index v4.3 ships, critics say benchmarks still overfit
CurieuxExplorer · x · 2026-09-08
Artificial Analysis released Intelligence Index v4.3:
- Terminal-Bench upgraded from 2.1 to 4.0, completing the migration to the latest version;
- Replaced τ³-Banking with AutomationBench-AA, an agentic workflow automation benchmark modeled on Zapier's business workflows with a private test set;
- Part of an ongoing rollout toward Intelligence Index v5.
A widely shared critique argues this is still not enough: many models score above their real performance due to overfitting; benchmarks should add creative tools (Blender, CAD, Adobe), real-world engineering problems, spatial reasoning and computer use, with private data. Example: most models can generate a 10-page PDF but fail or err heavily at 300 pages, and outputs are increasingly homogeneous with Opus.
Related event: Artificial Analysis Rolls Out Intelligence Index v4.3 with New Benchmarks(8 posts)→
More from Models
- OpenAI launches GPT-6.1 Terra: faster, cheaper computer-use model for the whole workday — airesearch12 · 2026-10-27
- GPT-6.1 Terra Imminent: gpt-6.1-terra Slug Appears in OpenAI Model Lists — koltregaskes · 2026-10-27
- In cyber, intention matters little: Chintala on model info leaks, and what users should do — brhydon · 2026-10-27
- GPT-6.1 Terra Model Slug Spotted in APIs and Codex Changelog Before Launch — testingcatalog · 2026-10-27
- Ortet's first health eval: 34 open-weight models, 50 tasks, generic rankings fall short — thegautamkamath · 2026-10-27
- VLM Run launches OCR Playground for side-by-side comparison of GLM-OCR, DeepSeek-OCR-2 and more — spillai · 2026-10-27