SJTU's SWAM trains world action models on 47,000 hours of video, hitting 40% SR on real robots
SJTU · hf · 2026-11-10
SWAM from Shanghai Jiao Tong University studies how to preserve language-video alignment when adding action learning to world action models. The team curated 47,000 hours of general, egocentric, UMI and robot demonstration data, systematically testing supervision mixes, annotation quality and architectures. Mixing egocentric video with UMI demos improves control generalization under a fixed budget. Post-trained only on RoboTwin Clean demos, SWAM reaches 88.10% (Easy) and 61.56% (Hard) success on RoboTwin, and 40.00% SR on three real-world tasks vs 23.33% for the strongest baseline. Data, code and weights are released.
More from Embodied
- SimEX: agents bootstrap robot skills in simulation, then fold towels in the real world — chris_j_paxton · 2026-11-10
- Amazon Ring unveils $250 autonomous home security drone that patrols and streams live video — Polymarket · 2026-11-10
- NVIDIA-led study: unblocking VLM gradients in VLA training lifts manipulation progress from 10.3% to 32.9% — stepjamUK · 2026-11-10
- Google DeepMind launches Gemini Robotics 2.5 with 96% accuracy sorting Mercedes-Benz parts — GoogleDeepMind · 2026-11-10
- SlashVibe is a physical desk controller for managing multiple running AI coding agents — Scobleizer · 2026-11-10
- Path Robotics raises $300M Series E at $2.45B valuation to expand physical AI — Rewkang · 2026-11-10