ALPS: a single visual encoder unifying state, language, and pixels for world models
cihangxie · x · 2026-10-23
Researchers from UC Santa Cruz, Google, and UCSB introduce ALPS, a visual state encoder for world models that unifies physical-state information, language alignment, and pixel reconstruction in one set of patch tokens, pretrained from scratch on images and text.
Key results:
- State: 0.829 relative LIBERO state-probe error, lowest among ViT-B encoders, matching the 3.5× larger V-JEPA 2-L.
- Language: 93.2 R@1 on Flickr30k image-to-text, within 2 points of the best image-text encoders.
- Pixels: 0.120 reconstruction ℓ1 error, lower than every compared encoder not trained to reconstruct.
- No compared encoder beats ALPS on all three.
Paper, code, and weights are released; the argument is a world model's visual foundation should support prediction, instruction, and inspection in one frozen encoder.
More from Embodied
- Peeling the iPhone Duo's nano-texture film reveals a surprisingly uneven screen underneath — film_girl · 2026-10-23
- XPeng IRON does voice-free handshakes, re-inferring motion every 0.2s via VLA 2.0 — CyberRobooo · 2026-10-23
- Viral humanoid robot cucumber-cutting video debunked: 2x background speed exaggerates the shaking — jonstephens85 · 2026-10-23
- Motoniq's self-reflective VLA hits 92% on Perturbed LIBERO, beating RHO (53%) and Claude agent (9%) — hbouammar · 2026-10-23
- OpenTac CAD auto-routes tactile sensor PCBs from a 3D model in 3 seconds — anand_bhattad · 2026-10-23
- Unitree's own robot conference matches top robotics gatherings in scale — micoolcho · 2026-10-23