Level-of-Token DiT Lets Pretrained Diffusion Models Use Arbitrary-Size Token Grids

GordonWetzstein · x · 2026-10-07

Researchers introduce Level-of-Token DiT, a minimal modification to pretrained DiTs that supports non-uniform token layouts:

The key idea generalizes the uniform token grid of pretrained DiTs into a "Level-of-Token" layout, where each token is a rectangle of any size and together they tile the whole image.
Each token is projected from the latent patches it covers; the DiT is conditioned on token shapes, and extent-dependent heads restore the asymmetric velocity, which is converted into the full-rank velocity.
They fine-tune both image (Flux.2) and video (Wan2.1) models with patch-wise asymmetric flow matching, preserving the pretrained generative prior.

More from Multimodal