UserArena benchmark uses 64,729 real shopping trajectories to test long-horizon agents

Jujia Zhao · hf · 2026-10-26

UserArena, from Leiden University, Renmin University, and Alibaba Group, is a real-world e-commerce interactive benchmark with 64,729 session trajectories from 11,010 users, covering 566,102 items and 13 action types.
It supports trajectory-level evaluation and offline validation against a controlled production A/B test, across five levels from output validity and local decisions to sequential behavior, personalization, and outcomes.
To improve simulator fidelity over long-horizon interactions, the authors propose On-Policy Trajectory Distillation.

More from coding & agent