SG-JEPA talk: latent dynamics consistency gives JEPA world models zero-shot physics generalization
Cohere · youtube · 2026-10-16
Yale PhD student Andy Zeyi Liu presented SG-JEPA at Cohere Labs Open Science Community: extending LeWorldModel by feeding the physics-governing parameter to the temporal model via action-conditioning and jointly training encoder and predictor through autoregressive latent rollout.
- Evaluation: dynamical tasks under different gravitational fields — same physical law but qualitatively different dynamics, from floating in weak gravity to rapid bouncing in strong — testing out-of-distribution generalization
- Results: vs DINO-WM, up to 2x lower open-loop prediction error on 2D datasets and up to 2.5x higher control success on 3D robotic datasets (with independently trained diffusion policies)
- Why it works: a linear feature model separating local law-conditioned error from recursive rollout amplification shows that backpropagating the multi-step rollout loss trains the encoder to keep features the dynamics depend on — most gains come from better representation, not better dynamics
More from Research
- 1-gram wireless mesoscope images cortex-wide activity in freely moving mice — KordingLab · 2026-10-23
- Reversible computing veteran poses open problem: universal BARCS primitives from unbiased Josephson junctions — MikePFrank · 2026-10-23
- Arc Institute uses genomic language model to map 376M prokaryotic noncoding sequences into 5.14M clusters — tedgreenwald · 2026-10-23
- The rise of AI-assisted peer review: reviewers are making it into the appendix — abursuc · 2026-10-23
- Anthropic uses Claude to formally verify security properties in 100+ open-source libraries at RISC-V assembly level — AnthropicAI · 2026-10-23
- Researchers test 9 strategies to pick LLM teams, beating random selection by up to 36.1% — shangbinfeng · 2026-10-23