Q-Planning: robot policies self-improve from 25% to 80% success, no new demos
rbhar90 · x · 2026-08-27
The demo-to-deployment gap in robotics is a data problem: the thread argues additional teleop SFT data on robot foundation models suffers from covariate shift, and ICL alone doesn't reach mastery. Q-Planning offers an alternative via test-time self-improvement.
- A frozen large visuomotor BC policy is paired with a small off-policy Q-function
- At inference, the policy samples N candidate action chunks; the Q-function scores them and a single-step Q-weighted average is executed
- Both successful and failed rollouts go into a replay buffer that fine-tunes only the Q-function; the BC policy is never updated
- On a hard fine-grained manipulation task, success rises from 25% to 80% in 100 robot attempts (30 minutes), with no extra human data
Essentially a "thinking mode" for robot policies via test-time compute.
Related event: Q-Planning Enables Robot Self-Improvement, Boosting Success from 25% to 80%(2 posts)→
More from Embodied
- China's AgiBot Reportedly Raising $210M at $7B Valuation, IPO May Slip to 2027 — xmercury_one · 2026-10-15
- The Verge: cute AI pet Eiliko uploads your conversations to Chinese servers — The Verge AI · 2026-10-15
- Sipeed MaixStation product page goes live for edge AI dev — xlltt · 2026-10-15
- Lukas Ziegler joins SPRIND Next Frontier Robotics Challenge jury, backs modular thesis for Europe — lukas_m_ziegler · 2026-10-15
- Gothic Church 3D Gaussian Splat Coming to Vision Pro, Hailed as Sharpest Ever — willeastcott · 2026-10-15
- Top-labs survey: 5 of 7 robot self-improvement studies show no cross-task gains, zero confirmed L5 — rohanpaul_ai · 2026-10-15