LIT trains VLAs to act before seeing: two-stage method fights visual shortcuts
DJiafei · x · 2026-09-11
Standard VLAs/WAMs train action experts directly on visual representations, but backgrounds, camera angles, or lighting in training data can correlate with demonstrated actions — the policy learns these visual shortcuts and fails when correlations shift in new scenes.
LIT (Latent Interface Training) restructures training into two stages:
- Learn to act first: train the action expert without images, conditioned on language, robot state, and each demonstrated action chunk's terminal end-effector pose.
- Learn to use vision: initialize from Stage 1, then jointly train with visual conditioning routed exclusively through a latent interface, supervised to reconstruct the same terminal pose — encouraging it to retain goal-relevant spatial information.
This constrains visual reliance while preserving goal-relevant spatial info, improving robustness out of distribution.
Related event: LIT Breaks Vision-Action Shortcuts in Robot VLA Models(4 posts)→
More from Embodied
- NVIDIA's ThinkWorld learns robot skills from unlabeled human videos via chain-of-dynamics reasoning — nvidia · 2026-10-28
- Just Action Model: decoupling what separates VLA from world-action models — Yonghao Yu · 2026-10-28
- NVIDIA Research releases Foundation4D: unified 4D perception model turns camera video into 3D scenes, no lidar needed — Scobleizer · 2026-10-28
- Dyna Robotics visits four hotel laundry rooms before starting Taku POC — JasonMa2020 · 2026-10-28
- Dyna engineer on Taku: from 24-hour napkin folding to full laundry workflows — JasonMa2020 · 2026-10-28
- Apple spent 20 years trashing touch-screen laptops — now it's making one — Scobleizer · 2026-10-28