Shanghai AI Lab's GameRealm: 120M frames of gameplay data for visual foundation models
Shanghai-AI-Laboratory · hf · 2026-11-19
Shanghai AI Laboratory, with Shanghai Jiao Tong University and Zhejiang University, released GameRealm, a large-scale dataset and paper grounding visual learning in interactive games:
- Over 120 million RGB frames and 1,000 hours of gameplay footage across all views.
- Synchronized metric depth, camera parameters, audio, control inputs, and 3D point trajectories; single-view, stereo, multi-view, and panoramic configurations.
- A unified collection pipeline combining human-operated interaction with agentic exploration scales acquisition across diverse game environments.
- Experiments show GameRealm consistently strengthens visual foundation models in both generation and geometric learning.
Authors include Weiguang Zhao, Yifan Wang, Junting Dong, Weinan Zhang, and Chunhua Shen.
More from Research
- Noam Brown confirms the math model claim; speculation points to an astra math fine-tune — haider1 · 2026-11-19
- NCME talk: generative psychometrics validates LLM-generated test items before human data — GolinoHudson · 2026-11-19
- Researchers say peer review has hit a breaking point as AI conference scale strains ICLR — LucaAmb · 2026-11-19
- KradleBench launches as a dynamic multi-agent benchmark pitting LLMs, world models and humans against each other — lmoroney · 2026-11-19
- KradleBench Launches: Minecraft Legend SethBling Dethroned by Model Astra 6.2 — fchollet · 2026-11-19
- Transformer Lab unveils Primus, a 'society of researchers' of AI agents, and is hiring — carsonfarmer · 2026-11-19