Sharpa's World Synesthesia Model for dexterous hands accepted at CoRL 2026
jiqizhixin · x · 2026-09-25
Dexterous hands are moving from demos to real applications: the key is not "can it move," but whether policies generalize stably under noisy depth, sparse tactile feedback, external perturbations, and unseen objects. Real sensor noise creates a vision gap — policies trained on clean observations fail in the wild.
Sharpa Robotics' World Synesthesia Model (WSM), accepted with high scores at CoRL 2026:
- Unifies visual geometry, tactile contact, proprioception, and action history into one reusable world model state, letting the policy continuously infer the hidden physical state of the hand-object system rather than matching current observations
- Built on a Dreamer-style RSSM for multimodal reconstruction, applying noise and dropout to depth inputs during training while reconstructing clean wrist depth — forcing the recurrent state to learn latent physics
The design directly targets sim2real and real-world deployment under sensor noise.
More from Embodied
- Robots & Beers meetup returns to Hong Kong Oct 15 as an Electronics Fair side event — broodsugar · 2026-09-25
- XPENG locks in suppliers for humanoid robots, targets 2027 deliveries — VraserX · 2026-09-25
- DeltaWAM cuts video model cost for bimanual manipulation, lifting RoboTwin success to 85.4% — Han Yan · 2026-09-25
- IFR report: 5M robots now working, China alone accounts for 59% of 2025 installs — lukas_m_ziegler · 2026-09-25
- Musk on China's humanoid robot games: the future will have 'a lot, a lot' of robots — XFreeze · 2026-09-25
- Meta sidesteps the VR/XR/MR jargon mess by just calling them VR glasses — pvncher · 2026-09-25