LeCun team's counterfactual world models push JEPA from "what happens next" to "what if"
TheTuringPost · x · 2026-11-21
TheTuringPost traces four years of JEPA world models and introduces a new paper by Yann LeCun, Randall Balestriero and Kerda Varaku on Counterfactual World Models.
Timeline of the JEPA lineage:
- Jun 2022: LeCun's position paper proposes predicting representations of what comes next, not pixels
- Jan 2023 I-JEPA: predict representations of masked image blocks from visible ones, no hand-crafted augmentations
- Feb 2024 V-JEPA: scaled to 2M videos; frozen ViT-H/16 hits 81.9% on Kinetics-400
- Nov 2024 DINO-WM: predicts future DINOv2 patch features from offline trajectories and plans with them
- Feb 2025 PLDM: JEPA trained on reward-free offline data for planning
- Jun 2025 V-JEPA 2 / 2-AC: pretrained on 1M+ hours of internet video; 2-AC post-trained on under 62 hours of robot video does zero-shot pick-and-place on Franka arms in two labs
The new paper's blind spot: a standard JEPA can only answer "what happens if I do this?"—not "what would have happened if I'd done something else?". With a boat pushed by a hidden current, the observed outcome tells you what the current did, but a standard JEPA asked a counterfactual returns the average again. The paper proves a model trained only on factual transitions cannot answer counterfactuals regardless of data or capacity; counterfactual world models instead estimate, in simulation, what an alternative action would have done under the same hidden disturbance.
Related event: LeCun and Colleagues Release Counterfactual World Models Paper(12 posts)→
More from Embodied
- Tesla FSD V15 visualizations go viral with detailed 3D night-time rendering — DimaZeniuk · 2026-11-22
- Tesla FSD visualization: from dots on a screen to photorealistic 3D night views in six years — XFreeze · 2026-11-22
- YQ Bionics' Y-Hand M2 packs 38 degrees of freedom as AI meets robotic manipulation — CurieuxExplorer · 2026-11-22
- Tesla FSD's photon reconstruction reportedly now sees better than human eyes in low light — XFreeze · 2026-11-22
- Kyros Labs builds Kepler, one workspace to replace 5-tool robot policy debugging — Repulsive-Fee-2735 · 2026-11-22
- Musk says Optimus will wash dishes and take out trash within 12-18 months — nikola_mr64990 · 2026-11-22