Probing 10 vision foundation models finds none supports a full spatial state
liuziwei7 · x · 2026-11-02
Researchers from NYU, NTU, HKUST and CUHK released "Probing Spatial State Representations in Visual Foundation Models", testing whether vision foundation models maintain a persistent spatial state — an internal record kept beyond observation, updated under self-motion, and used to act.
The paper decomposes spatial state into four levels: L1 spatial structure (reading the scene), L2 object persistence (remembering objects out of view), L3 predictive spatial update, and L4 actionability.
Using frozen-feature probing on 10 models, with trained probes predicting the 3D direction of previously seen objects on real ScanNet++ scans, the study finds no model performs well across all four levels — suggesting current vision foundation models lack the persistent spatial state needed for embodied intelligence. Paper, code and an interactive demo are public (arXiv:2610.40636).
More from Embodied
- Stanford's RoboArchitect Auto-Designs Robot Harnesses, Lifting Unseen-Scene Success from 9.7% to 43.3% — Shengguang Wu · 2026-11-02
- CausalWAM Isolates Action Causes in World Action Models via Sparse Routing — causal-intelligence · 2026-11-02
- CUHK's CaPtain-4B Post-Trains Small VLMs as Code-as-Policy Robot Agents, Hitting 78.3 on LIBERO-PRO — CUOHK · 2026-11-02
- Foxconn deploys 10+ humanoid robots in its Texas AI server plants, built on NVIDIA GR00T — CyberRobooo · 2026-11-02
- Unitree robot dog guards home under a 'Beware of Dog' sign while the backup dog naps — yassineyousfi_ · 2026-11-02
- JD's WorldBridge lifts robot task generalization to 70.7% with video plus sparse actions — Jun Cen · 2026-11-02