Artificial Analysis makes Time per Task its headline metric; default effort cuts task time 3-4x
ArtificialAnlys · x · 2026-11-05
Artificial Analysis announced the next version of its API performance benchmarking across 1,000 serverless endpoints tested multiple times daily, with Time per Task — how long a model takes to complete a task, rather than how fast it streams — as the new headline metric.
Key findings:
- Most leading labs set API defaults below max effort, sharply cutting time per task
- GPT-6 Astra: 52 on the Intelligence Index at medium effort vs 57 at max, but 5.8 min per task instead of 18
- GPT-6.1 Sol: 55→46 at 3.9 min instead of 20.0
- Claude Opus 5.5: 59→48 at 6.8 min instead of 26.0
Failed requests now score a speed of 0, with an Endpoint Error Rate shown per endpoint. Charts also contrast Max vs API Default effort on intelligence, time and cost per task (MiniMax-V2.5-Pro at $0.26/task vs Claude Opus 5.5 (max) at $12.99).
Related event: Artificial Analysis Revamps API Benchmarks Around Time per Task(2 posts)→
More from Models
- Ripple claims $32M compute for open-weight coding SOTA; skeptics question the math — teortaxesTex · 2026-11-05
- Gemini 4 Flash (codename Hydrogen) spotted A/B testing in AI Studio and LMArena — lyraxana · 2026-11-05
- Perceptron's new 3D hand model beats ACE-Ego-Hand by 12-25% on HOT3D and ARCTIC — gui_penedo · 2026-11-05
- 0.8B open-source HaloGuard 2.0 claims streaming-guard SOTA, beating models 5-10x its size — const_reborn · 2026-11-05
- MolmoAct 2 Takes Second to GPT-6 Astra in RoboColosseum's Real-World Robot Tests — DJiafei · 2026-11-05
- Claude Max subscriber at $100/month says Opus 5.5 'almost feels unlimited' — prasenx · 2026-11-05