DSReg post-hoc regularization provably recovers latents without reconstruction; Nature MI paper unifies superposition theory
ninamiolane · x · 2026-10-17
- Yujia Zheng et al. introduce DSReg, a simple regularization applicable post hoc to pretrained models that can provably recover individual latent variables of the true world even without reconstruction (e.g., JEPA-style training).
- A companion Nature Machine Intelligence paper by David Klindt, Nina Miolane and colleagues unifies neural superposition and sparse interpretable codes: synthesizing identifiability theory, compressed sensing, and quantitative interpretability into a three-step framework — networks trained for classification recover latent features up to linear mixing, followed by sparse coding and quantitative interpretation.
- The works are framed as complementary stages: the framework serves as a first stage, with SAEs or similar methods as a second stage to disentangle the rest and get axis-aligned representations.
More from Research
- APS Global Physics Summit opens Neural Systems session, abstracts due Oct 2026 — chklovskii · 2026-10-17
- RAX open-sourced: frozen coding agents learn robot tasks via online exploration, zero demos — du_yilun · 2026-10-17
- Open textbook "Elements of Robotics" by Ben-Ari & Mondada now freely available — burkov · 2026-10-17
- U. Tokyo's TwinX robot: one wire links two wheeled-legged bots for morphing and teamwork — LexiLove · 2026-10-17
- NVIDIA open-sources midtraining recipe: 500B tokens lift Nemotron agentic benchmark scores — denisyarats · 2026-10-17
- PySR v2 ships after two years of work: plugins, custom types, much faster — MilesCranmer · 2026-10-17