ZeroVLN navigates without SFT of the VLM brain using tool-calling and closed-loop RL

ZYT-AI · hf · 2026-11-20

ZeroVLN, from Beihang University, Zhuoyu Technology and Zhejiang University, is a slow-fast vision-language navigation framework that avoids navigation-domain SFT of the VLM brain. It uses a semantic tool-calling interface to convey movement decisions in natural language to an action expert, then jointly optimizes the brain's decisions and the expert's execution via closed-loop RL. It substantially outperforms SFT-based hierarchical VLN methods on VLN-CE while preserving the VLM's general capabilities, and is the first to jointly optimize slow-fast policies with agentic RL in VLN-CE.

Original post →

More from Embodied

Embodied channel →