Robotics policy keeps the vision backbone frozen and trains on a consumer GPU
mayfer · x · 2026-07-25
A robotics policy recipe that freezes a strong vision backbone, feeds all patch tokens into a small transformer policy, and uses a block-causal mask.
- No VLM or backbone fine-tuning is needed.
- The approach works with transformer-based policies such as VQ-BeT and Diffusion Policy.
- It is reported to train on a consumer GPU.
The attached diagram shows an observation trunk that encodes multi-view observations and goal inputs, then a policy head with frame-wise attention producing actions over time.
More from Embodied
- Humanoid walking is continuous correction — watch recovery, not locomotion, in demos — Olivier__OG · 2026-10-26
- CircuitHub unveils The Grid, a factory-scale robot making PCBs in the US twice as fast as Shenzhen — garrytan · 2026-10-26
- AI researchers guessed robots wouldn't outrun humans until 2034 — it already happened — ben_j_todd · 2026-10-26
- Shanghai AI Lab-led team releases 100-hour egocentric wearable dataset for humanoid training — ChuckDBrooks · 2026-10-26
- RealSense D585 Pro AI-Powered 3D Stereo Depth Camera Unboxed for Physical AI — chrismatthieu · 2026-10-26
- Physical world agent swarm demo shared on X — pduan · 2026-10-26