Transformer Transformer: design a full robot from one manipulation demo, 73% tracking error cut
SongShuran · x · 2026-09-05
Stanford's Huy Ha, Karen Liu, and Shuran Song introduce Transformer Transformer (CoRL 2026), a unified model for motion-conditioned robot co-design: feed it a manipulation demonstration and it generates a complete robot embodiment — links, joints, motors, inertial properties — optimized for that motion.
Key points:
- A diffusion transformer trained on RoboTokens, a unified tokenization of embodiments, states, and actions, spanning wheeled bimanual, quadruped, and humanoid spaces
- Dynamics Self-Guidance: reward-agnostic dynamics predictions are converted to task-specific values at inference, steering embodiment diffusion for zero-shot optimization of unseen rewards
- A fabricated cloth-flinging design on an ALOHA2 platform cut tracking error by 73% and max joint speed by 30% vs. the original, beating an evolutionary baseline
Paper, code, and a 17-minute walkthrough video are available.
More from Embodied
- Tesla's RIM process kills the paint shop, shrinking Cybercab factory footprint ~50% — elonmusk · 2026-09-05
- XDOF, 3 months out of stealth, in talks for Series B at $1.2B valuation — ZeYanjie · 2026-09-05
- AMD unveils Threadripper Halo Station: 96-core CPU plus MI350P cards with 288GB HBM3E — Aroochacha · 2026-09-05
- AirSim co-creator: we used to pay thousands of dollars to Unreal designers for training environments — sytelus · 2026-09-05
- 10,000 Robotics Trainer Jobs at $50-$90/Hour Opening Globally Within a Week — Exp_Mark · 2026-09-05
- Backed by Hillhouse in three rounds, Fullive.ai launches Somni sleep hardware to train a 'response model' — 创业邦 · 2026-09-05