Qwen team's E-Commerce Bench runs 18 frontier models through a simulated year; no model dominates
dair_ai · x · 2026-09-02
The Qwen team released E-Commerce Bench, which runs an agent through a simulated 365-day year operating several online stores at once, scoring 18 frontier models across seven dimensions.
- GPT-5.6 Sol earns the most, growing a 100,000 stake into 1,431,425, but ranks 16th of 18 on fraud avoidance and trails Fable 5 on operational efficiency.
- Among open-weight models, Qwen3.8-Max-Preview leads at 416,252, 38% above GLM 5.2 (high), and shows the strongest learning over the horizon — progressively bargaining suppliers down across repeated orders.
- No single model dominates every dimension. Worth reading for anyone evaluating agents beyond a single session.
Related event: Qwen Releases E-Commerce Bench: No Model Masters Year-Long Store Operation(4 posts)→
More from Models
- MOSS SFT vs Flux 3 Klein vs SD 3.0: Same-Prompt Comparison Exposes SD3's Age — ZootAllures9111 · 2026-10-18
- Quick take: GPT-6 matches Opus 5.5 at coding, far ahead at computer use — _vztu · 2026-10-18
- Open models now above Sol-level, outpacing expectations as community awaits V4.1 Pro — yacineMTB · 2026-10-18
- Codex took 3 days, Opus 5.5 nailed it in an afternoon — banned user's video protest — dotey · 2026-10-18
- Yacine MTB: Users will remember how Anthropic treated them once intelligence proliferates — yacineMTB · 2026-10-18
- Mistral's Timestral forecasting models start at 5M params and run on plain CPUs — lmoroney · 2026-10-18