SpatialClaw: Training-Free Code-as-Action Interface Boosts VLM Agentic Spatial Reasoning
CMHungSteven · x · 2026-09-29
A new arXiv paper from an NVIDIA internship team studies how the action interface shapes VLM spatial reasoning agents:
- Existing spatial agents use either single-pass code execution (locked into a strategy before seeing intermediate results) or structured tool-call interfaces (too rigid for freely composing operations).
- SpatialClaw is a training-free framework using code as the action interface: a stateful Python kernel preloaded with input frames and perception/geometry primitives, where the VLM writes one executable cell per step conditioned on all prior outputs — enabling flexible, open-ended 3D/4D spatial reasoning.
- The author invites agents/robotics/VLM eval researchers to reproduce the frontier-model numbers.
More from Research
- Why chirality matters for AI for Science: enantiomers behave differently in biology — madeofAjala · 2026-11-16
- RSS 2027 submissions open with FAQ clarifying two-stage review and negative-result bar — GeorgiaChal · 2026-11-16
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16
- Berkeley paper: LLMs answer with stale values even when the update is in context — ombre-ai · 2026-11-16
- LatamGPT releases 178.4M-document open Latin American corpus on Hugging Face — OmarUFlorez · 2026-11-16
- Open-source TotalSegmentator spots a 42mm dilated aorta on a routine chest CT — MaziyarPanahi · 2026-11-16