Four ETH Zurich lectures map embodied AI's gaps: hands, space, world models, data
le_james94 · x · 2026-10-13
James Le watched all four Frontiers of Embodied AI lectures at ETH Zurich (May 29, 2026, 3h17m total, recordings on YouTube) and breaks down each speaker's diagnosis of the field's missing piece:
- Jitendra Malik (UC Berkeley / Amazon): writing an omelet recipe is easy; cracking the egg is where robots stall — the hard problem sits below planning, in hand trajectories and applied forces. He also analyzes why teleoperation, human video, and simulation each fall short as training data.
- Vladlen Koltun: spatial cognition in frontier models; locates the gap in representation.
- Yann LeCun: world models.
- Shuran Song: making manipulation data easier to collect.
Le's own takeaway: two gaps in sequence — Koltun's is representation, Malik's is information. He disagrees with Koltun on one point: a simulated apartment is fine for space, but for contact it only supplies physics someone wrote down. "Space is on the screen. Contact isn't." He also probes whether world models trained on video learn anything about contact — none of the four lectures did.
Related event: ETH's Four Embodied AI Lectures: Malik, Koltun, LeCun Each Point to a Gap(3 posts)→
More from Embodied
- RT-SAFE: Agents Hit 94.1% Task Success but Only 0.7% Safe Success — Lianhuiq · 2026-10-13
- Tesla: The Only Robot Cars You Can Actually Buy — yunta_tsai · 2026-10-13
- NVIDIA takes RTX Spark laptop launch events global, with a Jensen-signed ASUS ProArt P14 up for grabs — NVIDIAAI · 2026-10-13
- Actuator speed, not just AI, is what's holding robots back from snappy actions — ayushthakur0 · 2026-10-13
- Stellantis demos Fiat 500e with Wayve's self-driving AI on Turin streets — alexgkendall · 2026-10-13
- Wayve CEO Stages at Wave to Push Autonomy into Fiat and Maserati — alexgkendall · 2026-10-13