UCSC's ALPS encoder unifies state, language and pixels for world models in one ViT-B
yuyinzhou_cs · x · 2026-10-23
UC Santa Cruz (with Google) introduces ALPS, a visual state encoder for world models that unifies physical-state information, language alignment and pixel reconstruction in one set of patch tokens, pretrained from scratch on images+text only (ViT-B, no video or robot data).
- State: lowest LIBERO state-probe error among ViT-Bs (.829), matching 3.5× larger V-JEPA 2-L
- Language: 93.2 R@1 on Flickr30k image→text with its own text tower
- Pixels: best reconstruction among non-reconstruction-trained peers (.120 ℓ1)
- Frozen-encoder planning: 85.5% mean LIBERO success (best in comparison); cross-view refinement cuts state-probe error 17% with ≤0.5 pt retrieval change
More from Embodied
- Formic raises $50M Series B to scale physical AI across 100+ US factories — garrytan · 2026-10-23
- fal starts fine-tuning a physical robot to draw, results next week — OdinLovis · 2026-10-23
- PEARL demo: an LLM agent that checks experiment results, revises plans, and re-tests — KevinKaichuang · 2026-10-23
- CU Boulder's ATLAS Institute Opens Tenure-Track Robotics Professorship — ryosuzk · 2026-10-23
- MiTaS multi-resolution tactile imitation learning hits 80% success vs 35.5% vision-only baseline — Jan_R_Peters · 2026-10-23
- Princeton's Action Self-Distillation Teaches Robots Language Navigation with Zero Human Labels — shahdhruv_ · 2026-10-23