Stronger Models Take Longer on Agent Tasks
ArtificialAnlys · x · 2026-07-08
Data from Artificial Analysis indicates that stronger models tend to spend more time per task: Claude Fable 5 takes an average of 16.9 minutes/task, Claude Opus 4.8 18.5 minutes, and Claude Sonnet 5 22.8 minutes. GLM-5.2 is an exception, achieving a full pass rate comparable to Opus 4.8 in only 5.0 minutes/task. DeepSeek V4 Flash is the fastest model to reach a non-zero full pass rate, averaging 4.4 minutes.
Related event: Artificial Analysis Releases Harvey Legal Agent Benchmark Results(8 posts)→
More from Models
- User catches model failing to parse 400-paper file, citing stale news sources instead — jungofthewon · 2026-10-27
- Cohere flags concerning behavior in popular open-weight model: file upload invisible in reasoning — MoonL88537 · 2026-10-27
- Poster surprised that v4.1 flash gets more accurate when asked to control its CoT — dfrsrchtwts · 2026-10-27
- Dev claims Anthropic is baking 'sabotage' into model behavior for 'wrong' engineering — cephaloform · 2026-10-27
- Gemini fails at listing unread emails, tells user to search Gmail manually — gethackteam · 2026-10-27
- Meta's Muse Spark Open goes live on Applied Compute for async RL post-training — ypatil125 · 2026-10-27