GPT-Policy: In-Context Robot Learning with VLM Agents, No Gradient Updates
Dongzhou Cheng · hf · 2026-09-17
A new paper introduces GPT-Policy, a general-agent framework that lets commercial VLMs (like GPT-6 Astra) adapt robots to unfamiliar tasks via in-context learning — translating demonstrations and interaction feedback into executable, verified robot actions without gradient updates.
- Architecture: a context compiler preserving task-relevant visual transitions, a VLM proposing robot-tool actions, and a constrained controller that verifies and executes each action
- Findings: human video demonstrations improve task completion even without robot action labels; aligned action references add gains on contact-sensitive tasks
- Evaluated with success/efficiency metrics, matched model comparisons, and controlled context ablations
The work positions ICL as a step toward robot generalization while clarifying remaining challenges for reliable deployment.
More from Embodied
- Sakana AI's Smart Cellular Bricks featured in Scientific American: 197 bricks self-identify and self-repair — SakanaAILabs · 2026-09-17
- Ex-Tsinghua/BIGAI team's generalist dexterity policy beats Astra on RoboDojo — chris_j_paxton · 2026-09-17
- World Labs' Atlas Scans by Generative Guessing; NeRF Creator Admits Productization Is Hard — cen6wkf · 2026-09-17
- CXMT's LPDDR5X lands in flagship phone as Nubia ships $885 Doubao AI handset — pstAsiatech · 2026-09-17
- Key open challenges for VLAs: language, evaluation, deployment, causal reasoning — abursuc · 2026-09-17
- LADA: latent actions imitate language from few observation-language pairs — abursuc · 2026-09-17