Transformer Transformer: design a full robot from one manipulation demo, 73% tracking error cut
SongShuran · x · 2026-09-05
Stanford's Huy Ha, Karen Liu, and Shuran Song introduce Transformer Transformer (CoRL 2026), a unified model for motion-conditioned robot co-design: feed it a manipulation demonstration and it generates a complete robot embodiment — links, joints, motors, inertial properties — optimized for that motion.
Key points:
- A diffusion transformer trained on RoboTokens, a unified tokenization of embodiments, states, and actions, spanning wheeled bimanual, quadruped, and humanoid spaces
- Dynamics Self-Guidance: reward-agnostic dynamics predictions are converted to task-specific values at inference, steering embodiment diffusion for zero-shot optimization of unseen rewards
- A fabricated cloth-flinging design on an ALOHA2 platform cut tracking error by 73% and max joint speed by 30% vs. the original, beating an evolutionary baseline
Paper, code, and a 17-minute walkthrough video are available.
Related event: Stanford's Transformer Transformer Auto-Designs Robot Morphologies(3 posts)→
More from Embodied
- XPeng commissions 80%-automated line for IRON humanoid, mass production by end-2026 — ingliguori · 2026-10-23
- HKU's TILDE Distills Embodied Foundation Models into Robot Policies — hkuhk · 2026-10-23
- 10 of 12 Beta Robots Worked Without Fine-Tuning — Do We Still Need It? — micoolcho · 2026-10-23
- EmbodiedEvo lets robot agents self-train, banking reusable Skills and Lessons across tasks — jiqizhixin · 2026-10-23
- KAIST's FRED lifts robot policy success from 19.4% to 50% via self-distilled future video representations — kaist-ai · 2026-10-23
- GALAX Launches World's First RTX 5080 With Dual 12V-2x6 Connectors at $3000 — chemist_slime · 2026-10-23