CUHK's CaPtain-4B Post-Trains Small VLMs as Code-as-Policy Robot Agents, Hitting 78.3 on LIBERO-PRO
CUOHK · hf · 2026-11-02
CUHK MMLab and NovaXbot present CaPtain, a pipeline that post-trains small open vision-language models into code-as-policy robot manipulation agents.
- Approach: Every tool is a low-level API down to IK motion control; the model writes code executed in a Python sandbox against low-level robot APIs—no frontier models or VLA backbones needed.
- Training: CaPtain-4B is trained with supervised fine-tuning plus multi-turn reinforcement learning using a physical critic.
- Results: It beats a Qwen3.5-48B baseline and its own SFT variant across LIBERO benchmarks—73.8 on LIBERO, 78.3 on LIBERO-PRO, 66.1 on LIBERO-Plus, 65.6 on LIBERO-MAX (vs. 54.4/59.5/48.6/45.6 for Qwen3.5-48B)—and is validated on real robots, RoboDoojo, and RoboTwin 2.0.
More from Embodied
- Stanford's RoboArchitect Auto-Designs Robot Harnesses, Lifting Unseen-Scene Success from 9.7% to 43.3% — Shengguang Wu · 2026-11-02
- CausalWAM Isolates Action Causes in World Action Models via Sparse Routing — causal-intelligence · 2026-11-02
- Foxconn deploys 10+ humanoid robots in its Texas AI server plants, built on NVIDIA GR00T — CyberRobooo · 2026-11-02
- Unitree robot dog guards home under a 'Beware of Dog' sign while the backup dog naps — yassineyousfi_ · 2026-11-02
- Probing 10 vision foundation models finds none supports a full spatial state — liuziwei7 · 2026-11-02
- JD's WorldBridge lifts robot task generalization to 70.7% with video plus sparse actions — Jun Cen · 2026-11-02