Geometric abstraction cuts robot agent VLM queries 52.8% and speeds manipulation 33%
siddkaramcheti · x · 2026-11-09
A CoRL 2026 Agentic Robotics Workshop paper asks whether agentic robot policies need to query VLMs on raw pixels at every step. The authors build a 3D Geometric Abstraction that grounds RGB-D into structured text and typed symbolic references, refreshed locally and read directly by LLM agents, invoking visual reasoning only on demand. On an online ring-on-peg task it cuts mean completion time 33.3%, model-request time 49.4%, recorded tokens 52.8%, and images per episode from 10.9 to 2.7 — showing explicit geometric state is an effective substrate for agentic robot policies.
More from Embodied
- Guava distills GPT-5.4 into a 4B robot agent with just 2,000 simulated trajectories — jbhuang0604 · 2026-11-09
- Dex-One2Many learns dexterous manipulation from one human video, 95%+ success across 4 hands — jbhuang0604 · 2026-11-09
- Fly-by-Code: coding agents pilot drones via active visual and physical feedback loops — jbhuang0604 · 2026-11-09
- μ0 world model learns robot physics from video only, beats π0 by 5 points on RoboCasa365 — jbhuang0604 · 2026-11-09
- NVIDIA unveils GR00T 2 open-world robot action model, hitting 95% success with 5 demos — CyberRobooo · 2026-11-09
- CoRL keynote to explore open-source data and hardware for robotics — micoolcho · 2026-11-09