LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation
Shaoan Wang, Aocheng Luo, Fei Huang, Jingyi Xu, Xiaoyang Wang, Yueyu Wang, Qianli Ma, Fan Yang, Ran Mei, Jia Wei, Jiangpeng Hu, Xuhao Liu, Hongming Chen, Yuanbin Shao, Yiyang Lin, Ziliang Li, Liang Pan, Xinhang Liu, Yuntao Ma, Tingxiang Fan
cs.RO, cs.AI
2026-08-31
LightNav-0 elicits Qwen3-VL-4B spatial priors with dual-channel pointing and three RVQ tokens, leading all 10 public sims on monocular success across VLN, object nav, and tracking.
Embodied navigation has to cover instruction following, object search, and tracking of moving targets, then still transfer across robots and scenes. Most systems still bolt on task-specific parts: waypoint predictors, topological maps, separate action heads. Perception, reasoning, and control stay fragmented, so open-vocabulary transfer is weak and the spatial priors already inside modern VLMs never reach the controller.
Models such as Qwen3-VL already ground objects, reason about space, and point on the image plane. Those skills are rarely elicited as robot control. LightNav-0's claim is direct: keep the pretrained backbone, add no per-task prediction head, and expose spatial intent through one token interface that is shared across tasks and embodiments.
The backbone is Qwen3-VL-4B-Instruct with an extended vocabulary only. Each decision emits a fixed five-token string:
The pointing prefix is an explicit spatial trace of constant length, so it does not pay the variable latency of free-form chain-of-thought. Because the grid lives in the image plane, the same codes transfer across robots. Reserved indices cover in-place turns, stopping, and an invisible target.
The action tokenizer uses three 256-entry codebooks. The first token is a coarse trajectory; the next two correct residuals. Any nonempty prefix is already executable, so a tight budget can stop at level 1 or 2. Full three-level decoding has 0.72 cm average displacement error, against 2.48 cm for a single-level K=4096 VADv2-style vocabulary.
Visual history is compressed by recency: recent frames keep higher sampling rate and finer resolution, older frames are pooled more aggressively, under a bounded token budget. Training has three stages. Embodied-reasoning mid-training produces LightNav-ER; supervised fine-tuning with DAgger follows; online GRPO then optimizes closed-loop rewards for arrival, efficiency, and tracking persistence. The corpus spans 2K+ scenes and 4K+ hours. In the SFT mixture, 77.6% of samples carry action supervision and 22.4% rehearse reasoning.
LightNav-ER scores 67.4 on the complete 8-benchmark embodied-reasoning average, above Qwen3-VL-4B at 63.1 and 8B Molmo2-ER at 62.8. Where2Place moves from 64.0 to 76.6; RefSpatial from 45.5 to 57.4.
The navigation checkpoint is monocular RGB, no depth, no odometry:
| Setting | LightNav-0 | Strongest monocular baseline |
| R2R SR / SPL | 68.5 / 62.8 | Qwen-RobotNav-4B 66.9 / 60.5 |
| RxR SR / SPL | 73.6 / 64.5 | Qwen-RobotNav-8B 73.4 / 63.5 |
| MP3D ObjectNav SR | 53.3 | CogNav 46.6 |
| HM3D v2 SR | 79.5 | FiLM-Nav 77.0 |
| EVT distracted-tracking SR | 82.6 | ReferTrack 73.3 |
| INSIGHT-Bench SR | 43.7 | JanusVLN 27.4 |
Dropping pointing supervision cuts R2R SR from 68.5 to 59.5 and HM3D v1 from 74.5 to 61.0. Initializing SFT from raw Qwen3-VL instead of LightNav-ER yields 65.8 R2R SR, 2.7 points lower. Real-world rollouts use the same checkpoint on four robot embodiments. Tracking labels contain only humans, yet the policy follows humanoid robots, wheeled robots, and carts.
A 4B monocular model with no task heads now leads all 10 public simulation settings on monocular success rate. Teams that want one policy for VLN, object search, and tracking do not need a panoramic stack or a depth camera for this recipe. Three RVQ tokens also keep exact token log-probabilities, so policy gradients apply without recasting a diffusion or flow-matching head as an MDP.
This is an alignment of existing VLM spatial skill onto control, not a new navigation architecture.
There is no standalone limitations section. The conclusion flags a single decision pathway with no split between high-frequency local avoidance and slower semantic planning, and open-world coverage still depends on curated embodied data; the authors suggest internet-scale video pretraining.
Panoramic methods still win some trajectory-fidelity numbers: RxR nDTW is 67.4, below DualVLN's 70.0. On INSIGHT-Bench, Institution scenes sit at 29.2 SR and extremum instructions at 37.2, a wide cell-to-cell spread. Real-robot results are qualitative; no public real-world success rates are given. Compute is not cheap: about 170 H100 hours for ER mid-training and 950 for SFT.