Accio Open-Sources CommerceAgentBench, Claude Passes 52%
dr_cintas · x · 2026-08-30
Accio open-sourced CommerceAgentBench, featuring 107 tasks rebuilt from real e-commerce workflows like buying, listing, ops, and fulfillment. It requires agents to handle emails, browsers, calendars, and logistics tools. While Claude Opus 5 leads, it only completed 66 of 107 tasks. The benchmark is derived from 10M SME users, 1.6M conversations, and 2,000 high-value workflows.
Related event: Alibaba's Accio Open-Sources CommerceAgentBench for E-commerce Agents(3 posts)→
More from coding & agent
- Simon Willison: code mode is the future of tool calling, if it can be sandboxed — springrod · 2026-10-16
- Dev workflow: Astra for tedious tasks, Claude for ideas not implementation — menhguin · 2026-10-16
- Agent named 'Ilya' reportedly scores 1.1pt higher on benchmarks — daniel_mac8 · 2026-10-16
- Gemini CLI fix: substring matching inlined unrequested binary files into model context — priyanshu1976 · 2026-10-16
- Building AI agents on real healthcare claims data: the domain work that makes them work — alex_verem · 2026-10-16
- OpenAI stealth-testing mysterious internal model 'Aeon' in Codex desktop app and Dots — cedric_chee · 2026-10-16