MINT in detail: 1,021-hour pseudo-label pipeline for egocentric camera and hand motion
CyberRobooo · x · 2026-09-08
The author shares full links and details for MINT: project page, paper, model, and open datasets are public.
- MINT jointly estimates camera motion, FOV, MANO hand parameters, and hand presence from egocentric RGB video in a unified 4-head model
- The multi-stage EgoPipeline generates pseudo-label supervision at scale, covering 1,021 hours of egocentric data
- A spatiotemporal backbone alternates frame-wise and global attention for appearance and motion
- Comparisons shown on rapid camera motion, close hand interaction, low light, bimanual coordination, and motion blur; failure cases (e.g., dough shaping) also published
- Collaboration between ShanghaiTech, Tsinghua, Wuji Technology, HKU, and Zhejiang University
Related event: WuJi Open-Sources MINT to Turn Egocentric Video Into Robot Training Data(3 posts)→
More from Embodied
- Bedrock Robotics raises $600M as its fully autonomous excavators do paid work on live job sites — Sethwinterroth · 2026-10-27
- AW-0 hits 80.8% on RoboTwin tasks, beating strongest baseline by 16 points — trevordarrell · 2026-10-27
- AW-0's code stream fixes world model prediction: picks mango where baseline grabs apple — trevordarrell · 2026-10-27
- AW-0 runs at 230ms on a single RTX 5090, 2.4x faster than autoregressive code planning — trevordarrell · 2026-10-27
- AW-0 debuts: one generative world action model plans in code, predicts, and controls robots — trevordarrell · 2026-10-27
- Skyfall AI's SKY-001 world model turns sketches into CAD builds in 3.9s, vs 199s for GPT-6 Astra — rohanpaul_ai · 2026-10-27