Artificial Analysis Rolls Out Intelligence Index v4.3 with New Benchmarks

On September 8, Artificial Analysis released Intelligence Index v4.3, another iteration on the path toward v5, involving two benchmark replacements and methodology adjustments that have sparked community debate over evaluation methods and leaderboard representativeness.

Confirmed

• Terminal-Bench was upgraded from v2.1 to v4.0, with 66 multi-step terminal tasks and the evaluation harness migrated from the Terminus 2 series to the latest version
• The 𝜏³-Bench Banking sub-benchmark was removed and replaced by the in-house AutomationBench-AA, built on Zapier business-workflow automation scenarios as a private test set
• Per methodology details posted by @ArtificialAnlys, category weights remain unchanged: Agents 30%, Coding 20%, General 30%, Scientific Reasoning and others as before
• The weight of private (non-public) test questions in the index rose to 45%
• According to @pbaylies, DeepSeek V4 Pro now ranks above Qwen 27B under the new version; @AccBalanced relayed that GLM-5.3-Flash climbed to third among open-weight models, performing notably well on complex agent tasks (both are reposted commentary; official leaderboard details prevail)

Why it matters

The v4.3 benchmark reshuffle means model scores and rankings on the leaderboard may shift. Terminal-Bench 4.0 and AutomationBench-AA better reflect real multi-step tasks and business-workflow automation, substantively strengthening agent capability assessment. Meanwhile, the private-question share rising to 45% reduces contamination and overfitting risk, per @inductionheads, but raises the bar for third-party reproduction. @CurieuxExplorer relayed critics' view that the leaderboard still fails to cover models' real capabilities — a focal point of community discussion.

2026-09-08 ~ 2026-09-08 · 8 related posts

• [source] Artificial Analysis Intelligence Index v4.3 raises private-question weight to 45% — ArtificialAnlys · 2026-09-08
• [source] Artificial Analysis rolls out Intelligence Index v4.3 with private AutomationBench-AA — inductionheads · 2026-09-08
• [source] Artificial Analysis Intelligence Index v4.3: Terminal-Bench 4.0, Zapier's AutomationBench — SteppenAxolotl · 2026-09-08
• Artificial Analysis updates Intelligence Index to v4.3 with two benchmark swaps — Profanion · 2026-09-08
• AA Intelligence Index v4.3: DeepSeek V4 Pro Edges Qwen 27B as Flash Next Beats Its Own MoE Flagship — pbaylies · 2026-09-08
• Artificial Analysis Intelligence Index v4.3 ships, critics say benchmarks still overfit — CurieuxExplorer · 2026-09-08
• GLM-5.3-Flash becomes 3rd strongest open-weight model as AA upgrades benchmarks — AccBalanced · 2026-09-08

1 near-duplicate retellings: SonglinYang4