Accio Open-Sources CommerceAgentBench, Claude Passes 52%
dr_cintas · x · 2026-08-30
Accio open-sourced CommerceAgentBench, featuring 107 tasks rebuilt from real e-commerce workflows like buying, listing, ops, and fulfillment. It requires agents to handle emails, browsers, calendars, and logistics tools. While Claude Opus 5 leads, it only completed 66 of 107 tasks. The benchmark is derived from 10M SME users, 1.6M conversations, and 2,000 high-value workflows.
More from coding & agent
- Militant releases AI music videos and dev tools — Kyrannio · 2026-08-31
- Reduce Claude Code Verbose Output with Open-Source Plugin — Any_Chemical9410 · 2026-08-31
- Optimal Settings for Llama.cpp + Qwen 3.8: n-max 4 Fastest — GodComplecs · 2026-08-31
- Beating Mem0, Zep, and Letta: BrainAPI on Agent Architecture Bottlenecks — shbong · 2026-08-31
- Context connectivity is the key to efficient AI agent workflows — brandon_galang · 2026-08-31
- AI Generated Code: Unverifiable Edge Cases and Inevitable Abstraction — eigenhector · 2026-08-30