COWBOY: sim-to-real framework gives mobile manipulators zero-shot whole-body grasping in the wild

rsalakhu · x · 2026-10-22

Ruslan Salakhutdinov's team unveiled COWBOY (CoRL 2026, arXiv:2610.28948), a scalable sim-to-real framework for contextual whole-body manipulation.

• A single generalist grasping policy coordinates mobile base, arm, dexterous hand, and actived camera end-to-end, adapting positioning, viewpoint, and grasps to object and scene geometry.
• Training: a vectorized environment-aware whole-body controller handles reaching across thousands of scenes; local RL experts are reused across full-scene layouts and distilled into one vision policy from point-cloud observations.
• Result: zero-shot transfer to hundreds of objects in cluttered real-world settings — tabletops, shelves, drawers, and other unstructured scenes.

Original post →

More from Embodied

• FLUX-mimic autonomous robot has been running nonstop for two days at GTC, team claims — robrombach · 2026-10-23
• OpenTac skin lets a Vega-1 robot sense pushes and comply with your nudges — anand_bhattad · 2026-10-23
• Google advances Coral NPU with open architecture, compiler tools, and commercial chips for on-device AI — GoogleAI · 2026-10-23
• Maxinsights Raises $30M Seed to Scale 2M Hours of Real-World Robot Training Data — CyberRobooo · 2026-10-23
• CoreWeave launches Physical AI Launchables: run Isaac Sim, CARLA, MuJoCo with one command — wandb · 2026-10-23
• iPhone Duo footage shows noticeable HDR artifacts resembling halation filter — flngr · 2026-10-22

Embodied channel →