DSReg: provably recovering individual world latents without reconstruction, fixing JEPA identifiability
illinois · hf · 2026-10-09
Prior methods for recovering individual world latents (nonlinear ICA, dictionary learning, causal representation learning) rely on reconstruction, auxiliary supervision, or distributional asymmetries; JEPAs identify latents only up to a linear transformation, leaving individual latents mixed.
DSReg (Dependency-Sparsity Regularization) closes the gap. Its key condition, Structural Diversity, requires that different latents leave distinct dependency footprints on observations. Building on LeJEPA's linear identifiability, the authors prove DSReg recovers individual world latents up to signed permutation — no reconstruction, decoder, or labels — establishing the first fully identifiable JEPA.
- Structural Diversity is strictly weaker than prior structural conditions
- Applies post hoc to any linearly identified representation, reusing checkpoints at no loss
- Validated on synthetic regimes, world model probes, visual encoders, and external renderers, improving latent recovery and downstream use
Related event: DSReg Provably Recovers Latent Variables Without Reconstruction(3 posts)→
More from Research
- Causal inner product from linear representation hypothesis validated across seven LLMs through 2026 — ChenhaoTan · 2026-10-09
- Solo Dev Pretrains 565M Hybrid LLM From Scratch on a Single RTX 4090 — BLUECOW009 · 2026-10-09
- BABA-is-AI: 2024 ICML benchmark that broke SOTA LLMs deserves a 2026 retest — moschles · 2026-10-09
- NVIDIA open-sources NV-Reason-CT, a native 3D vision-language model for CT scans — NVIDIA Developer · 2026-10-09
- One Epoch of Toloka's Enterprise RL Data Boosts Qwen3.5-27B Agent Benchmarks by up to 44pp — MParakhin · 2026-10-09
- srush builds Jax-Lean transpiler to formally verify JAX tensor code — srush_nlp · 2026-10-09