Artificial Analysis launches Coding Agent Index v2.0 with lower scores across the board
ArtificialAnlys · x · 2026-10-23
Artificial Analysis released Coding Agent Index v2.0, scoring configurations by benchmark and adding Cost per Task and token usage dimensions; methodology and version history are public.
Compared with v1.5, across 21 configurations measured on both versions the mean index score drops from 54.8 to 47.4, with 20 of 21 moving down. The shift is driven mainly by the new judge FrontierCode v1.1, where the same configurations average 45.1% versus 65.6% on DeepSWE v1.1, plus a smaller contribution from stricter SWE-At… judging.
The SWE-Atlas-QnA-AA dataset and judge configuration are open-sourced on Hugging Face.
More from coding & agent
- NVIDIA unveils OpenShell Reference Architecture to set boundaries for enterprise AI agents — NicoVerderosa · 2026-10-23
- Most 'unauthorized' AI agents in US bank accounts were let in by customers themselves, per Axios — nptacek · 2026-10-23
- Personal AI agents become a capital-intensive race as cloud execution environments reshape cost structures — sujingshen · 2026-10-23
- LLMs make code so cheap that recursive rewrites could make verified software tractable — jd_pressman · 2026-10-23
- Hugging Face hands OpenEnv RL environments project to Linux Foundation for neutral governance — lmoroney · 2026-10-23
- Theo writes the definitive explainer on what an AI coding "harness" actually is — pvncher · 2026-10-23