RECON paper: representation conditioning beats alignment, 25x faster than REPA
udmrzn · x · 2026-10-18
Two recent papers (RECON and Dino Forcing) reach a similar conclusion: representation conditioning outperforms representation alignment for diffusion transformer training.
How RECON works: REPA-style methods align an intermediate layer to a pretrained vision encoder via an MLP head that is discarded after training, leaving later layers supervised only by the velocity loss. RECON instead feeds vision encoder tokens into the later layers during training; at inference, with no clean image available, it reuses the MLP head as a proxy for the encoder and projects its output tokens into the later layers.
Results: adds zero parameters and converges far faster — 63x faster than SiT and 25x faster than REPA. Code is open source on GitHub.
Related event: RECON speeds up diffusion training 25x via representation conditioning(2 posts)→
More from Research
- Labs spend $10M+ of compute on math—why not on mechanistic interpretability? — Miles_Brundage · 2026-10-18
- Random layer-8 neuron matches specialized neuron at carrying AI personas — Sauers_ · 2026-10-18
- New interpretability technique clusters DPO-induced failure modes from preference data — AkshatS07 · 2026-10-18
- Goodfire essay: neural networks are already clean—polysemanticity was misread — rajistics · 2026-10-18
- OpenAI releases 700+ AI-generated math papers, with claimed progress on Riemann hypothesis — PolarBearby · 2026-10-18
- Gebru resurfaces TESCREAL paper arguing the AGI project is rooted in eugenics traditions — PolarBearby · 2026-10-18