Stanford's Level-of-Token Diffusion Cuts Video Gen Tokens, Speeds Up Generation Up to 4.6x
GordonWetzstein · x · 2026-10-07
Level-of-Token (LoT) Diffusion starts from an observation: diffusion models spend the same compute on a blank wall as on a face, yet you often know in advance where detail matters.
- Key idea: generalize the uniform token grid of pretrained DiTs into a multiresolution LoT layout — each token is a rectangle of any size tiling the image, with fine tokens where detail is needed and coarse tokens elsewhere.
- Method: minimally modifies a pretrained DiT — each token is projected from the latent patches it covers, the DiT is conditioned on token shapes, and extent-dependent heads restore the asymmetric velocity before conversion back to full-rank velocity. Fine-tuned on Flux.2 (image) and Wan2.1 (video) with patch-wise asymmetric flow matching, preserving the pretrained prior.
- Layout sources: bounding boxes, semantic masks, texture variance, and depth all work with one model; per-frame video layouts follow motion with 1.3–2.1× fewer tokens.
- Results: 6,632 tokens instead of 14,336 for one image — 2.5× faster generation, up to 4.6× when detail is concentrated. An agent can also paint an importance map directly (2.6× fewer tokens) or plan a rough Blender 3D scene for video (1.5× fewer tokens).
Related event: Stanford's Level-of-Token Diffusion Speeds Up Generation Up to 4.6x(4 posts)→
More from Multimodal
- Jordi Pons launches interactive AI music game 'A Chicken Dies' — jordiponsdotme · 2026-10-07
- LichtFeld Densification plugin cuts runtime from 85.5s to 16.7s on same GPU — janusch_patas · 2026-10-07
- Nano Banana 2.1 ships with higher-quality images, bug fixes, and lower prices — OfficialLoganK · 2026-10-07
- OmniReasoning: audio-visual joint reasoning benchmark lifts Qwen3-Omni by 12.8 points — _akhaliq · 2026-10-07
- Seedance 2.5 generates a 30-second Paris Halloween heist, upscaled 480p to 1080p — LudovicCreator · 2026-10-07
- Kling 4.0 Flash short: robot promises human cruelty is gone — but never says how — LudovicCreator · 2026-10-07