UPenn paper unifies diffusion and autoregression on one corruption lattice to predict decoding costs
upenn · hf · 2026-10-10
A UPenn team frames diffusion, autoregressive, and hybrid generative models as paths on a single corruption lattice, defining a schedule's cost as the dependence discarded by its parallel steps. They show the minimum steps of a zero-cost schedule are set by data geometry: for Markov-on-graph data, it equals the graph's treedepth (logarithmic in sequence length, linear in grid side length). Schedules below this bound pay positive cost, and the authors predict schedule rankings before decoding using a pairwise-dependence kernel estimated from pretrained weights, verified across text, image, and video generation. The work offers a unified design principle for future AR and diffusion models; code is open-sourced.
More from Research
- Jeremy Avigad's slides on the future of mathematics in the age of AI — ChengleiSi · 2026-10-10
- Tetris RL experiment: pretraining caps what RL can reach — PPO can provably converge to a bad policy — shizhediao · 2026-10-10
- Models trained on different data converge to similar concept geometry, researcher argues — cephaloform · 2026-10-10
- NULLs wins COLM Privacy & Security Workshop Best Paper for natively unlearnable LLMs — AdtRaghunathan · 2026-10-10
- Phantom Transfer: data poisoning survives 11 data-level defenses, NeurIPS 2026 paper shows — OwainEvans_UK · 2026-10-10
- DeepMind's Pushmeet Kohli on why AlphaFold didn't solve protein folding — Latent Space · 2026-10-10