Q-Planning: robot policies self-improve from 25% to 80% success, no new demos
rbhar90 · x · 2026-08-27
The demo-to-deployment gap in robotics is a data problem: the thread argues additional teleop SFT data on robot foundation models suffers from covariate shift, and ICL alone doesn't reach mastery. Q-Planning offers an alternative via test-time self-improvement.
- A frozen large visuomotor BC policy is paired with a small off-policy Q-function
- At inference, the policy samples N candidate action chunks; the Q-function scores them and a single-step Q-weighted average is executed
- Both successful and failed rollouts go into a replay buffer that fine-tunes only the Q-function; the BC policy is never updated
- On a hard fine-grained manipulation task, success rises from 25% to 80% in 100 robot attempts (30 minutes), with no extra human data
Essentially a "thinking mode" for robot policies via test-time compute.
Related event: Q-Planning Enables Robot Self-Improvement, Boosting Success from 25% to 80%(2 posts)→
More from Embodied
- SEAR: first large-scale controlled study of self-evolving robot agents, 7 LLMs, 330 tasks — xiye_nlp · 2026-10-16
- Podcast: subsea robotics startup Ulysses on operating 1,500m underwater and ocean carbon — blakemenezes · 2026-10-16
- Sony unveils $5,900 gold-plated Walkman with AI music enhancement — Polymarket · 2026-10-16
- ROBIB Meetup #3 returns to Ghent on Nov 19 with Physical AI talks, demos and panel — MarwaEldiwiny · 2026-10-16
- VLX-Go: Multi-Stage Vision-Language Training Boosts Robot Person-Following Among Distractors — omlab · 2026-10-16
- Ledger Atom Debuts as Agent-Approval Security Device; Critic Says a $100 iPhone 8 Beats It — adamamcbride · 2026-10-16