Alibaba Open-Sources CommerceAgentBench for Long-Horizon Agent Testing
mhdfaran · x · 2026-08-27
The Alibaba International team has open-sourced CommerceAgentBench, a benchmark designed to evaluate long-horizon agents. Unlike standard Q&A benchmarks, it places an agent in a simulated environment of real online services—such as an inbox filled with 300 messy procurement emails—and tasks it with making the right buying decisions. This setup tests the agent's ability to handle complex, stateful, and long-duration business workflows. The project is now open source and welcomes contributions for mock environments.
More from coding & agent
- Open-Source Pinloop Uses Claude Code to Auto-Screen Millions of Job Posts — thisdudelikesAI · 2026-10-15
- Kimi Releases Workflow Tutorial #3: Swarm, Plugins and Local Project Sync — Kimi_Moonshot · 2026-10-15
- Dev bills $20,000 for an app built with $200 of Claude Code — IgorCarron · 2026-10-15
- Dan Shipper: run product and labs in parallel, even with a labs team of one — danshipper · 2026-10-15
- Gemini 4 Argon wrote 5,000+ lines of pure Python to build its own ray-tracing 3D engine — aitrendz_xyz · 2026-10-15
- Minecraft fully ported from Java to Rust, file by file, server and client included — Vjeux · 2026-10-15