When hand tracking misses the plug-in moment: how should robot imitation data be evaluated?
Klutzy_Cap8492 · reddit · 2026-10-03
A Reddit post raises an evaluation question for robot learning data: when recording a human plugging in a cable, hand tracking can vanish exactly during insertion due to occlusion, leaving a pose-label gap where alignment turns into contact. Episode-level recall can stay high while missing the most critical short phase, and computing pose error only on successful detections hides the failure. The author points to MEgoVista's protocol (reporting precision/recall/F1 alongside reconstruction error and assigning error to missed detections) as a starting point, and proposes for manipulation data: coverage broken down by approach/contact/withdrawal, plus longest consecutive gap during contact. They ask the community what protocol they use to decide if episodes with missing contact-phase labels are still usable for imitation learning.
Related event: Brief hand-tracking gaps corrupt robot imitation learning data(2 posts)→
More from Embodied
- ETH's Anymal robot completes 10km autonomous alpine hike with VLM-guided ROAM system — ChongZzZhang · 2026-11-19
- iFixit tears down Apple HomeView: lots of glue and 'endless frustration' — Resident-Fall4351 · 2026-11-19
- Removing human video drops robot OOD success from 60% to 7%: generalization comes from human video — gui_penedo · 2026-11-19
- PSYONIC's prosthetic Ability Hand now used by 90+ robotics orgs, raising Series A — DhowCIO · 2026-11-19
- Simulator misspecification breeds overconfidence: sim-to-real study on SPEED+ — Nikolay Dolgov · 2026-11-19
- Doppel world model renders the robot body and beats action-only baselines across 8 embodiments — mangahomanga · 2026-11-19