MolmoAct2 Controls Robotic Arm Zero-Shot
DJiafei · x · 2026-07-10
MolmoAct2 is introduced as an action-inference model capable of running zero-shot on the SO-ARM101 robotic arm, emphasizing that it works out of the box "without any training."
It has now been integrated into LeRobot, covering the entire pipeline:
- Fine-tuning: Supports full-parameter or LoRA
- Evaluation: Provides a ready-to-use evaluation workflow
- Real-world deployment: Can be directly deployed onto physical robots
Technically, it combines the Molmo2-ER vision-language backbone with a flow-matching action expert, translating images, language, and embodiment states into action chunks. The post also details the hardware requirements: bf16 inference takes about 12GB of VRAM, LoRA fine-tuning can be done on a single 24GB GPU, and calibration corrections are already built into the checkpoint.
More from Embodied
- Humanoid walking is continuous correction — watch recovery, not locomotion, in demos — Olivier__OG · 2026-10-26
- CircuitHub unveils The Grid, a factory-scale robot making PCBs in the US twice as fast as Shenzhen — garrytan · 2026-10-26
- AI researchers guessed robots wouldn't outrun humans until 2034 — it already happened — ben_j_todd · 2026-10-26
- Shanghai AI Lab-led team releases 100-hour egocentric wearable dataset for humanoid training — ChuckDBrooks · 2026-10-26
- RealSense D585 Pro AI-Powered 3D Stereo Depth Camera Unboxed for Physical AI — chrismatthieu · 2026-10-26
- Physical world agent swarm demo shared on X — pduan · 2026-10-26