Q-Planning: frozen BC policy plus small Q-function enables robot self-improvement
animesh_garg · x · 2026-09-18
The Q-Planning paper (Garg lab, podcast coming soon) tackles a core BC limitation: policies can't self-improve from failures without new human demos, while RL fine-tuning doesn't scale to billion-parameter visuomotor policies.
- Attach a small off-policy Q-function to a large frozen BC policy; Q estimates value, so it absorbs both successful and failed deployment rollouts
- Inference: Q-weighted single-step average over BC action-chunk draws; self-improvement fine-tunes only Q, BC weights untouched
- Results: 10 iterations lift LIBERO-10 from 93% to 99% and RoboTwin from 83.8% to 91.4%; the same loop improves on two contact-rich bimanual real-robot tasks with zero human intervention
Related event: Q-Planning Enables Self-Improving Robot Policies via Small Q-Functions(3 posts)→
More from Embodied
- New probe tests 10 vision foundation models on persistent spatial state across 4 levels — liuziwei7 · 2026-11-02
- Wearing Whoop and Apple Watch on a single dual-slot band — tinyfool · 2026-11-02
- REA: async brain-cerebellum runtime for robots where policies never wait on the LLM — Pengfei Zhou · 2026-11-02
- Robot Combat Sports Coming to Japan with a Permanent Arena in Tokyo — cixliv · 2026-11-02
- REK robot fighting in the cage: humanoid bot in boxing gloves draws a crowd — broodsugar · 2026-11-02
- iPhone Duo: a folding iPhone mod impresses, but its hinge raises crackgate concerns — rudrank · 2026-11-02