Accio Open-Sources CommerceAgentBench for Real Agent Tasks
kimmonismus · x · 2026-08-28
Accio has open-sourced CommerceAgentBench, comprising 107 e-commerce tasks across procurement, listings, and operations. Unlike standard benchmarks, it evaluates agents based on actual changes, saves, or submissions made across browsers, emails, and APIs, rather than just their claimed reasoning steps.
Related event: Alibaba's Accio Open-Sources CommerceAgentBench; Top Score Just 61.7%(6 posts)→
More from coding & agent
- Decision models match LLMs on routing and safety — and are 7-8x faster — randal_olson · 2026-10-16
- Temporal Serverless Workers on Cloud Run: queue-driven autoscaling, scale to zero — rseroter · 2026-10-16
- Meta's Superintelligence Lab Runs 200+ Direct Reports—A Lesson for Flat Multi-Agent Systems — peterjliu · 2026-10-16
- LangChain CEO: the most interesting agent harness work lives at the harness-infra boundary — hwchase17 · 2026-10-16
- GitSwarm paper: agents share branchable Git memory, solving all 30 IMOProofBench problems in one run — anirudhg9119 · 2026-10-16
- Hands-on with Gemini 4 Argon: faster than Opus 5.5 but hallucinates more and costs more — prasenx · 2026-10-16