StarVLA's VLAct trains VLAs on 16 GPUs by reshaping action representations, not data scaling
jiqizhixin · x · 2026-09-16
StarVLA's VLAct (Beyond Data Scaling: Representation-Centric Continued Pre-training for VLAs) argues VLA progress can come from better representations instead of more data — crucial since robot data can't be scraped and every trajectory requires real hardware and teleoperation.
- Continued pre-training uses only 16 GPUs on open-source data.
- Key technique: an Orthogonal Finetuning (OFT) head that directly shapes the backbone's action representations.
- Evidence matters: an OFT head improves from 61.7% to 75.8%, but swapping the same backbone to a PI head loses the gains — the improvement lives in the representation, not the weights alone.
More from Embodied
- Wearing Whoop and Apple Watch on a single dual-slot band — tinyfool · 2026-11-02
- REA: async brain-cerebellum runtime for robots where policies never wait on the LLM — Pengfei Zhou · 2026-11-02
- Robot Combat Sports Coming to Japan with a Permanent Arena in Tokyo — cixliv · 2026-11-02
- REK robot fighting in the cage: humanoid bot in boxing gloves draws a crowd — broodsugar · 2026-11-02
- iPhone Duo: a folding iPhone mod impresses, but its hinge raises crackgate concerns — rudrank · 2026-11-02
- InflatableBots: University of Calgary's inflatable robots bring dynamic haptics to large-scale VR — lukas_m_ziegler · 2026-11-02