RealReplicaBench: All AI Agents Fail E-commerce Test, Top Score 56.1
APPSO · wechat · 2026-08-17
APPSO reports on RealReplicaBench, an e-commerce AI agent benchmark by Alibaba's AccioWork. Across 107 real business tasks, all models failed, with top scorer Claude Opus 5 at 56.1. The benchmark emphasizes task completion, requiring results usable by next steps, by replicating real environments and verifying final states. The team plans to update tasks for model evaluation and optimization.
Related event: AI models fail real-world work benchmarks as e-commerce agents fall short(2 posts)→
More from Research
- ImageNet should retire: DiT method rankings negatively correlate between ImageNet and text-to-image — YouJiacheng · 2026-10-16
- GAE paper learns a geometry-native latent space for 3D-consistent world generation — yshan2u · 2026-10-16
- Agents turn conservative as context grows: edits shrink from 150 to 30 lines — dbreunig · 2026-10-16
- LMR-BENCH at EMNLP 2025 tests whether LLM agents can reproduce real LLM research — ZhiyuChen4 · 2026-10-16
- Ex-OpenAI researcher claims most neurons are legible now; critics call it hype — anshulkundaje · 2026-10-16
- Physicist pushes back on ML community's acceptance that models may be beyond understanding — tszzl · 2026-10-16