Berkeley's CLIFT enables closed-loop fine-tuning for closed-source humanoid models
keerthanpg · x · 2026-08-20
UC Berkeley, in collaboration with Google DeepMind and NVIDIA, published "CLIFT," a method for Closed-Loop Iterative Fine-Tuning that does not require access to internal model parameters (gradients, losses).
Problem & Motivation:
- Closed-weight robot foundation models (e.g., Gemini Robotics On-Device) are generalists but lack task mastery for agile, contact-rich humanoid tasks due to covariate shift.
- Mastery requires closed-loop learning, but closed-source models only expose a managed SFT API, making standard RL (like PPO) impossible.
Core Method:
- The key insight is that reinforcement signals can be encoded directly into supervised training data.
- By using only the SFT API as a black-box operator mapping observation–instruction–action chunks to a tuned policy, the system achieves RL-style policy improvement without modifying the model or training procedure.
Results:
- This method successfully transforms a small, on-device model based on open-source Gemma into a humanoid specialist, outperforming some of the largest action models on manipulation tasks.
More from Embodied
- Team says no technical blockers remain for v1 self-replicating factory using off-the-shelf parts — ihorbeaver · 2026-10-16
- Skip the fancy vision: a $300 automatic screw feeder beats LLM-controlled picking — ihorbeaver · 2026-10-16
- Author's own caveat: direct LLM robot control only suits repetitive, constrained tasks — ihorbeaver · 2026-10-16
- Precise part insertion via LLM-written data collection code and a linear regression aligner — ihorbeaver · 2026-10-16
- Picking parts from a random pile with an extra flip-and-align platform — ihorbeaver · 2026-10-16
- Robot fully controlled by frontier LLMs assembles gadgets autonomously for 1.5 hours error-free — ihorbeaver · 2026-10-16