Maniformer paper cuts egocentric hand tracking error to 4.29mm for robot learning data
heyshrutimishra · x · 2026-10-12
Maniformer, a physical AI data platform, published a new paper tackling the key bottleneck of egocentric robot datasets: 3D hand pose accuracy. Validated against a 26-camera mocap lab, finger error drops to 4.29mm and head error to 0.65-2.83mm.
Key points:
- Teleoperation is slow and costly (machine, operator, env reset per hour), while cameras on people scale with headcount — egocentric datasets have grown from hundreds to thousands of hours.
- Video only shows what was done; converting to robot actions requires metric 3D hand positions via inverse kinematics and retargeting, so supervision quality depends on the underlying hand pose.
- The paper presents a full pipeline (tri-fisheye depth, 3D hand reconstruction, full-frame trajectory estimation) optimized with observation consistency, anatomical plausibility and temporal continuity constraints.
More from Embodied
- Robotics Needs a New 'Capture' Technology: Why Kinematics Data Is the Missing Scaling Input — andrew_n_carr · 2026-10-13
- Collov Labs launches NewEyes AI, a personal agent powered by smart glasses — SimplyAnnisa · 2026-10-13
- Robotics research goes big: one work reportedly used 64 H200 GPUs — chris_j_paxton · 2026-10-13
- Norway Proposes Temporary AI Glasses Ban in Schools, Gyms and Other Public Spaces — thione · 2026-10-12
- He built a cat activity dashboard with Opus 5.5 by sniffing Ring, Petlibro and Whisker app traffic — indragie · 2026-10-12
- Maker's 3D-printed robot comes to life with servo legs and custom frame — shekitup · 2026-10-12