NVIDIA-led study: unblocking VLM gradients in VLA training lifts manipulation progress from 10.3% to 32.9%
stepjamUK · x · 2026-11-10
A popular VLA training recipe blocks the action expert from updating the VLM underneath it. Asher Hancock, Marco Pavone and a team from NVIDIA, Princeton and Stanford varied two factors independently: whether the VLM also predicts discrete action tokens, and whether the expert's gradients flow back into it.
Findings:
- In open-loop driving, blocking gradients costs little when action tokens are on
- With tokens off, letting gradients through cuts trajectory error by 25%
- Closed loop is where it matters: trained together from scratch, gradient flow lifted driving route progress from 0.600 to 0.703, even with action tokens on
- In manipulation, a π0.5 re-implementation went from 10.3% to 32.9% task progress without FAST, and 19.5% to 30.7% with it; MolmoAct2 improved in the same direction
Conclusion: the blocked-gradient default deserves reexamination.
More from Embodied
- NVIDIA Ships Alpamayo 2 Edge: 2B-Parameter Onboard AV Reasoning at 160-190ms — nvidia · 2026-11-11
- NVIDIA launches open Alpamayo 2 Edge for real-time in-vehicle AI reasoning — nvidia · 2026-11-11
- NVIDIA open-sources GR00T 2 robot foundation model with 100K hours of training data — DrJimFan · 2026-11-11
- Stanford DroneHacks: 100 students fly GPS-denied PX4 drones on stereo cameras in 48 hours — mrpollo · 2026-11-10
- UT Austin open-sources OopsieVerse, a damage-aware safety benchmark for household robots — DynamicWebPaige · 2026-11-10
- Memories.ai pays users to record real-world tasks for robot training data — testingcatalog · 2026-11-10