Paper finds diffusion language models encode denoising progress in hidden states
Sapienza · hf · 2026-07-23
Subliminal Clocks studies whether diffusion language models encode denoising progress internally.
- The authors show that DLMs contain a latent representation related to the diffusion timestep inside residual streams
- Probes can extract this signal across layers, suggesting denoising progress is decodable from activations
- Steering the model along a low-dimensional subspace tied to the inferred timestep changes confidence and entropy in predictable ways
- The paper also analyzes the geometry of this representation and argues it is structured and interpretable
Related event: Study: Diffusion Language Models Have Subliminal Clocks(2 posts)→
More from Research
- Alex Townsend posts 200 open problems in numerical linear algebra for humans and AI agents — IgorCarron · 2026-09-11
- Navier-Stokes, Riemann, P vs NP: what this week's math buzzwords mean for you — koltregaskes · 2026-09-11
- Fruit fly brain as an LLM: connectome-driven language model demo goes live — ngxson · 2026-09-11
- Harry Collins: LLMs can't do frontier science because they can't invent new language — whoamisri · 2026-09-11
- The Waymo effect: how AI is quietly making research less collaborative — JohnHammersley · 2026-09-11
- Causal-only attention for non-generative tasks is wasteful, argues HF engineer — antoine_chaffin · 2026-09-11