Artificial Analysis opens up its Coding Agent Index: 39 configs, 3 benchmarks, full methodology
ArtificialAnlys · x · 2026-10-21
Artificial Analysis published the full leaderboard and methodology behind its Coding Agent Index, covering 39 agent×model configurations with composite score, time per task, and per-task API cost on a log scale.
The index equally weights three benchmarks — DeepSWE v1.1 (113 SWE tasks), Terminal-Bench 4.0 (66 terminal tasks), and SWE-Atlas-QnA (124 technical Q&A) — averaging pass@1 over three attempts per task. The site cautions that similar index scores can hide different strengths across repo tasks, terminal workflows, and rubric-based evals.
More from coding & agent
- GameForge plugin lets Hermes Agent build playable games and playtest them itself — Teknium · 2026-10-21
- Tired of re-authing every new AI tool? Use a gateway like executor — verrsane · 2026-10-21
- Community builds 4 GUIs for the Pi coding agent as official client stalls — solyarisoftware · 2026-10-21
- Dev uses AI to upgrade Laravel app from 5.8 to 11 in two days — hermanschutte · 2026-10-21
- Nace.AI Launches Drex Max 2.0, a 27B Local Decision Model for Computer Use — DotaMate · 2026-10-21
- 'The purge of tests': theo is Thanos-snapping 50% of his tests away — vincent_koc · 2026-10-21