CV legend Jitendra Malik: robots are wasting 3D structure signal, and real2sim2real deserves human credit
JitendraMalikCV · x · 2026-09-21
The argument
Berkeley professor Jitendra Malik reflects on 3D/4D reconstruction:
- It's one of computer vision's great achievements, rooted in photogrammetry and multi-view geometry (Hartley & Zisserman), fused with deep learning over the last decade; recognition-derived shape priors (e.g., HMR for humans) and recent systems like VGGT and SAM 3D deliver impressive results.
- Much robotics work in the learning era ignored 3D structure, which he calls "wasting valuable signal" since robots operate in the 3D world.
- He welcomes the rediscovery of real2sim2real via Astra, citing years of projects from his own group, and asks that credit go to human researchers, not just AI models.
Related event: Jitendra Malik: Robots Discarding 3D Structure Are Wasting Valuable Signals(3 posts)→
More from Embodied
- Walden Robotics' Sage model quadruples robot uptime on Toyota factory lines — adnothing · 2026-11-12
- Robotics debate: matching training to test distribution is not the best approach, says researcher — EpisodeYang · 2026-11-12
- Deployment is where physical AI grows up — and its most important data source, per Tedrake's CoRL talk — m_wulfmeier · 2026-11-12
- Runway teases triple bet: real-time avatars, interactive worlds, robot action models — c_valenzuelab · 2026-11-12
- WICL: robots learn manipulation in-context, 83.4% success on all 50 RoboTwin tasks — _akhaliq · 2026-11-12
- After CoRL, robotics feels far from solved: Gemini 4 Carbon evaluated — DJiafei · 2026-11-12