Project Kitchen Turns VR Gameplay Into Scalable Robot Manipulation Training Data
zhenjun_zhao · x · 2026-09-17
Project Kitchen is a VR-based gamified egocentric data collection platform that makes robot data collection engaging and hardware-independent, enabling crowdsourced large-scale deployment.
- Motivation: Generalizable manipulation policies need large diverse datasets, but real demonstrations are costly; existing approaches are tied to specific robot hardware or suffer from incomplete annotation.
- Method: Inspired by game design that sustains engagement, the platform elicits diverse goal-directed manipulation in VR. Game2Policy extracts embodiment-invariant affordance cues (contact points and sub-goal states) from gameplay trajectories.
- Results: An affordance model pre-trained on game data is jointly fine-tuned with downstream policies using only a handful of real-robot demonstrations, bridging the game-to-real gap.
The work offers a gamified route to robot-free scalable data collection.
More from Embodied
- Sakana AI's Smart Cellular Bricks featured in Scientific American: 197 bricks self-identify and self-repair — SakanaAILabs · 2026-09-17
- Ex-Tsinghua/BIGAI team's generalist dexterity policy beats Astra on RoboDojo — chris_j_paxton · 2026-09-17
- GPT-Policy: In-Context Robot Learning with VLM Agents, No Gradient Updates — Dongzhou Cheng · 2026-09-17
- World Labs' Atlas Scans by Generative Guessing; NeRF Creator Admits Productization Is Hard — cen6wkf · 2026-09-17
- CXMT's LPDDR5X lands in flagship phone as Nubia ships $885 Doubao AI handset — pstAsiatech · 2026-09-17
- Key open challenges for VLAs: language, evaluation, deployment, causal reasoning — abursuc · 2026-09-17