LeCun co-authors AD-E2E-JEPA: world model drives end-to-end without any driving policy
ylecun · x · 2026-09-30
A new paper (arXiv 2609.34085) by Haoran Zhu, Wancong Zhang, Yann LeCun and Anna Choromanska asks whether a world model can drive without training a driving policy.
- It systematically evaluates existing action-conditioned JEPA world models (LeWM, DINO-WM, JEPA-WM) for end-to-end autonomous driving, using a goal-conditioned zero-shot planning setup with ground-truth future observations as goals to isolate world-model quality from policy learning.
- Existing models trade off accuracy vs. compute: accurate models are expensive, efficient ones can't plan well.
- AD-E2E-JEPA adds a SIGReg-regularized learnable projector over projected patch embeddings, cutting planning patches 16x and embedding dimension 4x for a 100x inference speedup — 0.8s for an 8-frame rollout over 256 candidate trajectories.
- Without any trained policy, the world model alone reaches goals 20m away within 4.0/2.8m displacement using trajectory vocabularies. Paper and code are public.
More from Embodied
- NVIDIA's HumanoidMimicGen turns one teleop demo into thousands of humanoid demonstrations — AjayMandlekar · 2026-09-30
- Another glowing CyberCab ride: whurley shares second robotaxi experience — whurley · 2026-09-30
- Laser on a 3D printer turns Kapton tape into laser-induced graphene with PWM control — johnowhitaker · 2026-09-30
- Muse Charm Puts an AI Agent on Your Keychain — and It Feeds on Your Data — BenBajarin · 2026-09-30
- Chaining MiniMax H3 keyframe clips for an arcade racer without perspective drift — NoMarionberry9951 · 2026-09-30
- Robot Policy Metadata Conditioning Fails: Text Prompt Completely Ignored — DominiqueCAPaul · 2026-09-30