ETH Zurich's EgoPHI estimates 3D hand-object contact and force from one egocentric image
cholz · x · 2026-09-10
Christian Holz's group at ETH Zurich presents EgoPHI (ECCV 2026), the first vision-based model to jointly estimate dense contact maps and 3D force distributions on hand and articulated object meshes from a single egocentric RGB image plus object geometry.
- Approach: registers hand meshes to iteratively refine object pose under occlusion, fuses image features with 3D geometry, and uses graph attention plus cross-attention to model intra-mesh geometry and hand-object relations, predicting per-vertex contact and 3D force direction.
- Data: a physics-simulation pipeline augments ARCTIC and H2O with dense per-vertex contact+force supervision, addressing the lack of scalable force ground truth; tests cover in-distribution and out-of-distribution benchmarks.
- Sim2real: instrumented cube and cylinder objects captured dense contact and force data from eight participants to evaluate transfer.
- Paper, code, and dataset are released.
More from Embodied
- DIY omnidirectional robot packs LIDAR, thermal camera and hot-swappable battery — arnie_hacker · 2026-09-10
- GPT-6 Astra crushes HumanCLAW embodied benchmark, interaction success nearly triples — LINJIEFUN · 2026-09-10
- Dev praises Apple's new health app and Watch HRV focus, says he'll ditch Ōura — TejasKumar_ · 2026-09-10
- KaRMA: first metric to quantify robot hand dexterity from kinematics alone, at IROS 2026 — pulkitology · 2026-09-10
- Tesla Robotaxi miles up 2.6x in six weeks to 1M as it recruits Cybercab fleet buyers — elonmusk · 2026-09-10
- EgoPHI method details: graph attention over 3D geometry for per-vertex contact and force prediction — cholz · 2026-09-10