LeCun's world model pitch: predict representations, not pixels, and plan via MPC-style cost minimization
le_james94 · x · 2026-10-13
lejames94 recaps LeCun's ETH world-model lecture: predict the next state given an action, then plan by minimizing a cost — "very classical stuff in optimal control going back to the 1960s" (model predictive control). What's new: it predicts representations, not pixels.
His slide explains why: pixel-level prediction averages incompatible futures into a blur, while JEPA maps encode(x)+action to encode(y), letting the encoder drop whatever can't be predicted. The catch is collapse — an encoder outputting a constant makes prediction trivial — addressed with regularizers like SIReg. The planning loop: perceive, propose actions, predict, score (cost zero = done), optimize.
Related event: LeCun on world models: predict representations, not pixels, plan via MPC(2 posts)→
More from Embodied
- RT-SAFE benchmark: real-time embodied agents crash 12.3× more, safe success drops to 0.7% — Lianhuiq · 2026-10-13
- RT-SAFE: Agents Hit 94.1% Task Success but Only 0.7% Safe Success — Lianhuiq · 2026-10-13
- Tesla: The Only Robot Cars You Can Actually Buy — yunta_tsai · 2026-10-13
- NVIDIA takes RTX Spark laptop launch events global, with a Jensen-signed ASUS ProArt P14 up for grabs — NVIDIAAI · 2026-10-13
- Actuator speed, not just AI, is what's holding robots back from snappy actions — ayushthakur0 · 2026-10-13
- Stellantis demos Fiat 500e with Wayve's self-driving AI on Turin streets — alexgkendall · 2026-10-13