MINT in detail: 1,021-hour pseudo-label pipeline for egocentric camera and hand motion
CyberRobooo · x · 2026-09-08
The author shares full links and details for MINT: project page, paper, model, and open datasets are public.
- MINT jointly estimates camera motion, FOV, MANO hand parameters, and hand presence from egocentric RGB video in a unified 4-head model
- The multi-stage EgoPipeline generates pseudo-label supervision at scale, covering 1,021 hours of egocentric data
- A spatiotemporal backbone alternates frame-wise and global attention for appearance and motion
- Comparisons shown on rapid camera motion, close hand interaction, low light, bimanual coordination, and motion blur; failure cases (e.g., dough shaping) also published
- Collaboration between ShanghaiTech, Tsinghua, Wuji Technology, HKU, and Zhejiang University
Related event: WuJi Open-Sources MINT to Turn Egocentric Video Into Robot Training Data(3 posts)→
More from Embodied
- LLMs' best robotics role: generating controllers to pretrain robot models — igilitschenski · 2026-09-09
- Robot simulation startup Antioch raises $32M Series A led by Greylock — HeyAmit_ · 2026-09-09
- Deft Robotics launches unified deployment platform for physical AI — Scobleizer · 2026-09-09
- Running software on a 60Hz e-ink panel: the anti-glass-slab hardware aesthetic — Ben_Reinhardt · 2026-09-09
- Astra drives a SO-101 arm pick-and-place in MuJoCo sim with zero policy training — shekitup · 2026-09-09
- Robotics researcher argues GPT-6 understands the physical world; Chris Paxton notes the cycle — chris_j_paxton · 2026-09-09