Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation
Hongyan Feng, Sunlai Chen, Xuanyu Liu, Miao Pan, Yangfan Xie, Yuxiang Cui, Zhongxiang Zhou, Rong Xiong, Wenqi Zhang, Jianwei Yin, Yueting Zhuang, Xuhong Zhang
cs.RO
2026-08-18
Navigation as pointing: the VLM picks a 2D pixel that projects to 3D, reasons only at key nodes, and trains with two-level GRPO; 66.2% SR on R2R-CE, 16.6s per task, 60% zero-shot on a real robot.
Dropping a VLM into embodied navigation hits three walls. The action space is misaligned: prior methods emit atomic actions like "turn left 30 degrees" or regress 3D coordinates directly, while VLM priors live in 2D image-text pairs, so forcing 3D geometry through the model produces spatial hallucinations and poor sample efficiency. Reasoning schedules are rigid: triggering chain-of-thought every step or at fixed intervals buys decision quality at real latency cost. And long-horizon memory is stuck between keeping every visual feature (attention dilution, context overflow) and discarding history indiscriminately (losing critical information), with no explicit spatio-temporal coordinates to organize what remains.
The Zhejiang team splits the fix into four pieces:
The base is Qwen2.5-VL-7B: one SFT epoch on the 90k-trajectory MultiNav-CoT, then 800 GRPO steps, roughly 760 GPU-hours in total.
| Setup | Metric | Result |
| R2R-CE val-unseen | success rate | 66.2% (prior best NavFoM 61.7, DualVLN 64.3) |
| RxR-CE val-unseen | success rate | 65.7% (DualVLN 56.9) |
| Single A800, per task | inference time | 16.58s (StreamVLN 37.47s, DualVLN 41.46s) |
| Long-horizon subset (>12.5m) | success rate | 49.8% (DualVLN 41.9%, StreamVLN 30.9%) |
| Real robot, zero fine-tuning | success rate | 60% (StreamVLN 49%, DualVLN 53%) |
Ablations price each component: swapping pixel actions back to NavFoM-style metric waypoints costs 24.8 SR points, the largest single term in the table; dropping local advantages costs 4.1 points and removing annealed guidance another 2.8; full history instead of anchor memory costs 4.3 points, and removing STI tokens another 2.6. The reasoning schedule is learned, not designed: after RL, reasoning on straight corridors falls from 38% to 11% of steps, and a 26.3% CoT ratio comes within 0.6 points of dense CoT (66.8%). Depth noise at sigma 0.2 costs only 2.8 points. Code is released under ZJU-OmniAI.
The design principle of letting a VLM do what its pretraining already taught it is backed hard by numbers: a 7B model outruns larger specialized systems on both benchmarks, cuts interaction steps from roughly 30 to 9, and halves inference time. The selective-reasoning finding generalizes to any long-horizon agent: reasoning density can be a trained reward rather than a hand-set schedule. And the memory template (expensive features only at anchors, lightweight geometric tokens for trajectories in between) is a copyable pattern for context-budget management.
Training leans on privileged simulator signals (progress, success, SPL, collision); the paper states plainly that online RL on a physical robot is unsupported, and real-world deployment is zero-shot transfer of a frozen policy. The depth dependency is structural: pixel projection, STI encoding, and coordinate transforms all need it, and the controlled-noise study stops at sigma 0.2, leaving gross depth failure and SLAM drift unassessed. STI encodes planar position and heading but not height, so multi-floor buildings are ambiguous. The reasoning-density reward thresholds (0.4/0.6) are tuned to R2R-CE distributions and would need retuning elsewhere. CoT distillation used Gemini 2.5 Flash, and the teacher-sensitivity experiment shows Qwen2.5-VL-7B self-generated CoT performs below no CoT at all, so the student's reasoning ceiling is set by the teacher. The SOTA claim covers val-unseen only, and compared methods do not share sensing and low-level control; the paper concedes the real-robot comparison is of complete systems in their recommended configurations.