Alibaba's HappyWorld-Bench: 1,138 video cases test world model reliability
alibabagroup · hf · 2026-09-24
Alibaba released HappyWorld-Bench on Hugging Face, a comprehensive benchmark arguing world models must be judged not just on generation quality but on consistency and responsiveness as agents explore, interact with, and modify generated worlds.
- Framework: a hierarchical capability taxonomy (W1-W6) across three tracks — video, spatial, and embodied world models.
- Scale: 1,138 video prompts, 300 spatial scenes, 254 embodied cases; a HappyWorld-Arena runs human A/B comparisons producing model-level Elo ratings alongside automated behavioral-correctness metrics.
- Coverage: 14 video world models, 9 spatial systems, 8 embodied candidates evaluated.
- Findings: reliability gaps everywhere — video models lose consistency on long rollouts and revisits; spatial models top out at 70.14% placement accuracy and 73.33% edit execution; embodied models struggle to preserve state across multi-step actions and respond precisely to altered action conditions and physics.
More from Research
- Ross Girshick posts CVPR slides recounting 1.5 decades of object detection and its link to the LLM era — CatAstro_Piyush · 2026-11-11
- What it takes to squeeze performance: EMA, cosine scheduling and custom dead-cluster revival — chrisemoody · 2026-11-11
- Trainable JWST model boosts exoplanet sensitivity 2–15x in Nature paper — bravo_abad · 2026-11-11
- Ben Todd's alt log-log plot shows task horizons hitting a singularity in under 12 months — ben_j_todd · 2026-11-11
- Eva Jablonka's concept links aphantasia to the evolution of language — AnnaCiaunica · 2026-11-11
- BioAgent-DF: silent hardware faults can alter AI science agents' decisions invisibly — INTOSYN-AI-Labs · 2026-11-11