LeCun's new paper: JEPA world models answer "what if" with no retraining
TheTuringPost · x · 2026-11-21
Randall Balestriero, Yann LeCun and co-authors released Counterfactual World Models: JEPAs That Reason About "What If". The core claim: world models trained only on factual transitions provably cannot answer counterfactual questions on their own. But for most physical systems, the counterfactual outcome is, to first order, the observed outcome plus the difference between the model's predictions under the alternative vs. factual action — so no counterfactual data, extra parameters, or retraining is needed.
Validated from pixels with frozen encoders (DINOv2, MAE):
- Counterfactual R² jumps from 38–46% to 86–96% across four simulators, same world models
- Underwater vehicle BlueROV2: 90–94%, with 16x more factual data
- Works with DINOv3, SigLIP 2, CLIP, EVA-02 encoders too: 81–95%
- Hindsight planning (finding the action that would have reached the goal): solves 35–69% of events vs 14–23% for an oracle planner
Theory includes an O(h²) error bound, a matching minimax lower bound, and failure conditions at discontinuities. Practical payoff: separating bad decisions from bad luck when debugging robots. Code and demo are available.
Related event: LeCun and Colleagues Release Counterfactual World Models Paper(12 posts)→
More from Research
- Lean community certifies Riemann proof; honeypot fake keys expose breaches at three AI providers — LeviTurk · 2026-11-22
- Riemann hypothesis 'proven by Turkish mathematician using AI' claim mocked as the 'Organic Turk' — CatAstro_Piyush · 2026-11-22
- Andriy Burkov releases The Hundred-Page Language Models Book, hands-on with PyTorch — burkov · 2026-11-22
- Generative AI meets cardiology: Dagstuhl seminar bridges ML and clinical practice — StephanMandt · 2026-11-22
- SubRoformer hits SOTA music source separation with DSP subband filters — affige_yang · 2026-11-22
- Emily Riehl releases Lean game teaching dependent type theory at intro-to-proofs level — abhi9u · 2026-11-22