AutomationBench: GPT-5.5 vs Claude Opus 4.8 Work Styles
ArtificialAnlys · x · 2026-07-07
Models exhibit significantly different work styles on AutomationBench. GPT-5.5(xhigh) leans towards dense operations, averaging 49 tool calls across 25 rounds per task. Claude Opus 4.8(max) is more deliberate, concentrating 35 tool calls within 14 rounds and recording fewer guardrail violations (0.55 vs. 0.66 per task).
Grok 4.3(high) uses the fewest rounds (13), but underperforms compared to models that persist to the end, as it tends to declare tasks complete prematurely rather than actually finishing them efficiently.
Related event: AutomationBench-AA Launches to Evaluate AI Agents on SaaS Workflows(7 posts)→
More from Models
- MissionReport benchmark: only 3 of 540 AI-generated government reports pass all criteria — Exp_Mark · 2026-10-27
- Local beats cloud: Qwen 3.8 FN handles private student data as well as frontier models — dr_lm · 2026-10-27
- Aidan Gomez: model's CoT said no tool calls right before making them — aidangomez · 2026-10-27
- Nat Lambert: Cohere's model is narrow RL training on internal infra ill-suited for open weights — natolambert · 2026-10-27
- OpenAI teases DOTS is coming to Europe 'soon' — mark_k · 2026-10-27
- Clifford Pickover challenges AI to decipher a message he encoded in Norse runes — pickover · 2026-10-27