VLX-Go: Multi-Stage Vision-Language Training Boosts Robot Person-Following Among Distractors
omlab · hf · 2026-10-16
Researchers at Om AI Research and Zhejiang University's Binjiang Institute present VLX-Go, a paper tackling embodied visual tracking: following a designated person among distractors, where prior methods show steep success drops (41.54% for the earlier OmTrekVI-A model).
Key points:
- A vision-language waypoint policy built on the hypothesis that visual and language representations should align with the tracking motion being learned
- Combines frozen vision encoders, multiscale temporal context, a transformer backbone, and a lightweight head for robot-centric waypoint prediction
- Three-stage training: vision-language alignment, multimodal reasoning mid-training, and tracking specialization on 19,532 oracle episodes
- On-policy refinement further improves robustness against distractors
The staged multimodal approach substantially improves robust person-following in crowded environments.
More from Embodied
- Viral meme nails AI's irony: it can write symphonies but can't do the dishes — briannekimmel · 2026-10-16
- WEDGE turns navigation agents' own failures into targeted training worlds — mohitban47 · 2026-10-16
- Knödel Watch: animated vocabulary cards on ESP32-S3, open-sourced 4 days after unboxing — alexcovo_eth · 2026-10-16
- SlipSense fuses pressure and vibration for 23ms zero-shot robotic slip detection — micoolcho · 2026-10-16
- Stanford's Fei-Fei Li team open-sources OpenWAM, a unified world-action model framework for robots — jiqizhixin · 2026-10-16
- Robot folding leaderboard: GPT-6 Astra leads at 1349 Elo, Muse Wonder trails at 1034 — qinzytech · 2026-10-16