LeCun and Colleagues Release Counterfactual World Models Paper
Between Nov 19–21, Yann LeCun, Randall Balestriero, and Kerda Varaku (Brown / NYU / AMI Labs) released and presented "Counterfactual World Models: JEPAs That Reason About 'What If'", giving JEPA world models guaranteed counterfactual reasoning: imagining what would have happened under a different action while keeping the same realization of external disturbances (e.g., a gust of wind). The method uses only offline data with no retraining, lifting counterfactual prediction R² from 38% to 96%.
Confirmed
- The paper is by Balestriero, Varaku, and LeCun, introduced by the authors themselves; TheTuringPost ran a multi-post thread placing the work in the four-year evolution of JEPA world models.
- The paper targets a core blind spot of standard world models—they only predict the average outcome. TheTuringPost illustrates with a ship pushed by a hidden current: before acting the model can only give an average ending, and when asked "what if I had steered differently", a standard JEPA just returns the average again, discarding the disturbance information; it is proven unable to answer counterfactual questions.
- Method: Balestriero stresses that a plain JEPA predictor or a trained stochastic predictor does not suffice; the hard part is imagining counterfactual outcomes under the same realization of exogenous variables. The paper formalizes ẑ{t+1}(a') = z{t+1} + μ(Ht,a') − …, adding a correction term to the observed next state. Per TheTuringPost, the fix is surprisingly simple: exploiting that the action–disturbance interaction is second-order in time, "two forward predictions plus one addition" yields the counterfactual result; the paper proves error optimality, with counterfactual accuracy reaching 86–96%.
- Experiments cover multiple simulated environments including HandReach and Ball-in-Cup; combined with hindsight planning, success rates across five tasks rise to as high as 69%, all using only offline data without retraining.
Why it matters
- Most current world models only do conditional prediction and cannot rigorously answer counterfactual questions; this work provides a guaranteed definition of counterfactual reasoning and a simple, implementable algorithm—a step toward world models with causal imagination, with direct relevance to planning and offline policy evaluation.
2026-11-19 ~ 2026-11-21 · 12 related posts
Primary sources
- New Paper from LeCun et al.: JEPA World Models Answer 'What If' Questions Without Training — burny_tech · 2026-11-19
- [source] LeCun co-authors method giving JEPA world models counterfactual reasoning with provable guarantees — randall_balestr · 2026-11-20
- LeCun and Balestriero show JEPAs can reason counterfactually from offline data alone — randall_balestr · 2026-11-20
- Correction term is key to counterfactual reasoning in JEPA world models — randall_balestr · 2026-11-20
- Counterfactual world models with hindsight planning hit up to 69% success across sim tasks — randall_balestr · 2026-11-20
- LeCun's Counterfactual World Models paper boosts JEPA prediction R² from 38% to 96% — ylecun · 2026-11-20
- LeCun and AMI Labs propose Counterfactual World Models: JEPA reasoning about 'what if' without retraining — udmrzn · 2026-11-20
- [source] LeCun team's counterfactual world models push JEPA from "what happens next" to "what if" — TheTuringPost · 2026-11-21
- Standard JEPAs provably can't answer counterfactuals, however much data they get — TheTuringPost · 2026-11-21
- Counterfactuals with two predictor calls and one addition, paper proves O(h²) optimal — TheTuringPost · 2026-11-21
- Two predictor calls fix world-model counterfactuals: R² jumps from 38–46% to 86–96% — TheTuringPost · 2026-11-21
1 near-duplicate retellings: TheTuringPost