Shuran Song's UMI recipe: handheld gripper + GoPro, then add sound, force and hands one channel at a time
le_james94 · x · 2026-10-13
lejames94 recaps Shuran Song's data-collection approach from ETH: capture manipulation data with a handheld gripper and a GoPro (UMI), pairing observations with end-effector actions so one policy transfers across an arm, quadruped, or drone after swapping the controller.
Her tradeoff: teleoperation isn't simple enough (specialized hardware per embodiment), human video too simple (no actions, no contact forces). She then adds channels one at a time: sound (ManiWAV, a few-dollar contact mic via camera audio), force (UMI-FT, a quarter-sized 6-axis sensor, 2g, under $11 in materials), and hands (DexUMI, wearable exoskeleton with joint encoders, 86% average success across 4 tasks).
Related event: Song Shuran: robotics data bottleneck is reusability, not quantity(2 posts)→
More from Embodied
- RT-SAFE: Agents Hit 94.1% Task Success but Only 0.7% Safe Success — Lianhuiq · 2026-10-13
- Tesla: The Only Robot Cars You Can Actually Buy — yunta_tsai · 2026-10-13
- NVIDIA takes RTX Spark laptop launch events global, with a Jensen-signed ASUS ProArt P14 up for grabs — NVIDIAAI · 2026-10-13
- Actuator speed, not just AI, is what's holding robots back from snappy actions — ayushthakur0 · 2026-10-13
- Stellantis demos Fiat 500e with Wayve's self-driving AI on Turin streets — alexgkendall · 2026-10-13
- Wayve CEO Stages at Wave to Push Autonomy into Fiat and Maserati — alexgkendall · 2026-10-13