AI2's MolmoMotion hits NeurIPS Highlight: 3D point-trajectory forecasting lifts robot pick-and-place to 76.3%
k7agar · x · 2026-10-03
MolmoMotion, a 3D point-trajectory forecasting model from AI2, UW and UNC, was selected as a NeurIPS 2026 Highlight paper with everything open-sourced.
- MolmoMotion-1M: the largest dataset of action-described, object-grounded 3D point trajectories, built from 1.16M videos
- MolmoMotion-4B: a 4B language-conditioned model that predicts each queried point's future 3D trajectory in a metric world frame from a short RGB history plus an instruction, handling rigid, articulated and deformable motion; SOTA on PointMotionBench
- PointMotionBench: a human-verified benchmark spanning 111 object categories and 61 motion types
Downstream: fine-tuned on DROID real-robot videos, initializing a manipulation policy with it lifts pick-and-place success from a Molmo 2 baseline of 56.0% to 76.3%; its trajectories also serve as explicit motion-control signals for image-to-video generation, beating the base model on all five motion metrics.
More from Embodied
- Removing human video drops robot OOD success from 60% to 7%: generalization comes from human video — gui_penedo · 2026-11-19
- PSYONIC's prosthetic Ability Hand now used by 90+ robotics orgs, raising Series A — DhowCIO · 2026-11-19
- Simulator misspecification breeds overconfidence: sim-to-real study on SPEED+ — Nikolay Dolgov · 2026-11-19
- Doppel world model renders the robot body and beats action-only baselines across 8 embodiments — mangahomanga · 2026-11-19
- FloMo: a world action model using 3D motion as the bridge from video pre-training to robot control — CSProfKGD · 2026-11-19
- MIT's paperclip-weight flying robot is first microrobot with onboard sensing and compute — MIT News AI · 2026-11-19