Sharpa's World Synesthesia Model for dexterous hands accepted at CoRL 2026
jiqizhixin · x · 2026-09-25
Dexterous hands are moving from demos to real applications: the key is not "can it move," but whether policies generalize stably under noisy depth, sparse tactile feedback, external perturbations, and unseen objects. Real sensor noise creates a vision gap — policies trained on clean observations fail in the wild.
Sharpa Robotics' World Synesthesia Model (WSM), accepted with high scores at CoRL 2026:
- Unifies visual geometry, tactile contact, proprioception, and action history into one reusable world model state, letting the policy continuously infer the hidden physical state of the hand-object system rather than matching current observations
- Built on a Dreamer-style RSSM for multimodal reconstruction, applying noise and dropout to depth inputs during training while reconstructing clean wrist depth — forcing the recurrent state to learn latent physics
The design directly targets sim2real and real-world deployment under sensor noise.
More from Embodied
- NYU's Steertuning Lets a Robot Learn a Chore From 5 Minutes of Human Demo — CyberRobooo · 2026-11-11
- Boston Dynamics Atlas demoed with Blender 5.3 — TomLikesRobots · 2026-11-11
- Lamport Robotics posts robot 'wake-up test' demo video — R4nsack · 2026-11-11
- Embodied AI Faces Two-Order-of-Magnitude Data Gap as Industry Heads to 10M Hours — GaryMarcus · 2026-11-11
- NVIDIA hosts CoRL 2026 meetup on scaling robot training with Isaac Lab — m_wulfmeier · 2026-11-11
- Lighthouse extends Obsidian welding foundation model to run on any robotic platform — bennash · 2026-11-11