GPT-Policy: In-Context Robot Learning with VLM Agents, No Gradient Updates
Dongzhou Cheng · hf · 2026-09-17
A new paper introduces GPT-Policy, a general-agent framework that lets commercial VLMs (like GPT-6 Astra) adapt robots to unfamiliar tasks via in-context learning — translating demonstrations and interaction feedback into executable, verified robot actions without gradient updates.
- Architecture: a context compiler preserving task-relevant visual transitions, a VLM proposing robot-tool actions, and a constrained controller that verifies and executes each action
- Findings: human video demonstrations improve task completion even without robot action labels; aligned action references add gains on contact-sensitive tasks
- Evaluated with success/efficiency metrics, matched model comparisons, and controlled context ablations
The work positions ICL as a step toward robot generalization while clarifying remaining challenges for reliable deployment.
Related event: GPT-Policy: Robots Learn New Tasks via VLM In-Context Learning(2 posts)→
More from Embodied
- Wearing Whoop and Apple Watch on a single dual-slot band — tinyfool · 2026-11-02
- REA: async brain-cerebellum runtime for robots where policies never wait on the LLM — Pengfei Zhou · 2026-11-02
- Robot Combat Sports Coming to Japan with a Permanent Arena in Tokyo — cixliv · 2026-11-02
- REK robot fighting in the cage: humanoid bot in boxing gloves draws a crowd — broodsugar · 2026-11-02
- iPhone Duo: a folding iPhone mod impresses, but its hinge raises crackgate concerns — rudrank · 2026-11-02
- InflatableBots: University of Calgary's inflatable robots bring dynamic haptics to large-scale VR — lukas_m_ziegler · 2026-11-02