JHU's Track, Articulate, Act reconstructs articulated objects and dexterous interactions from casual monocular videos
mangahomanga · x · 2026-09-17
Researchers at Johns Hopkins' Brains, Bots, and Behavior Lab introduce Track, Articulate, Act, a real-to-sim framework that reconstructs simulation-ready articulated objects (doors, drawers, cabinets, laptops, ovens) from a single casually captured monocular RGB video — no RGB-D, multi-view input, prior scans, manually specified joints, or robot demos needed.
Key insight: dense 3D point tracks are an embodiment-agnostic articulation cue — points on the fixed link stay stationary while points on moving links follow coherent revolute or prismatic motion. The pipeline repurposes pretrained models (SAM 3D for geometry, mesh segmentation, 3D scene flow), segments links, estimates joints and state trajectories, reconstructs the articulated asset, re-targets recovered 3D hand motion, and replays the interaction in MuJoCo physics simulation.
More from Embodied
- New probe tests 10 vision foundation models on persistent spatial state across 4 levels — liuziwei7 · 2026-11-02
- Wearing Whoop and Apple Watch on a single dual-slot band — tinyfool · 2026-11-02
- REA: async brain-cerebellum runtime for robots where policies never wait on the LLM — Pengfei Zhou · 2026-11-02
- Robot Combat Sports Coming to Japan with a Permanent Arena in Tokyo — cixliv · 2026-11-02
- REK robot fighting in the cage: humanoid bot in boxing gloves draws a crowd — broodsugar · 2026-11-02
- iPhone Duo: a folding iPhone mod impresses, but its hinge raises crackgate concerns — rudrank · 2026-11-02