Hierarchical Diffusion Transformer hits 1.48 FID on ImageNet 256 in end-to-end pixel-space generation
niloofar_mire · x · 2026-11-19
New arXiv paper 'Hierarchical Diffusion Transformer for End-to-End Image Generation' (Xiaoyu Wu et al., with Lydia E. Kavraki and Steven Wu):
- Problem: pixel-space diffusion learns representation and generation end-to-end without a pretrained encoder, but plain pixel Transformers trail two-stage latent diffusion. The authors attribute part of the gap to keeping one token per patch throughout the backbone, with no explicit visual hierarchy.
- Method: HiT progressively groups related tokens into single group tokens processed at the Transformer bottleneck, then ungroups back to image positions; a compression budget tunes average compression ratio, with group counts varying across images and diffusion timesteps.
- Emergent structure: the model groups coherent objects and parts without segmentation labels, and aligns caption words with matching image patches without region annotations.
- Results: 1.48 FID on ImageNet 256×256, comparable to leading latent diffusion models. Paper, code, and demo are available.
More from Multimodal
- Open-source Claude Code plugin turns any song into a finished music video automatically — AIandDesign · 2026-11-19
- Procedural 3D will be the backbone of video generation, not pure video models — dreamwieber · 2026-11-19
- Seedance 2.5 demo: a 'museum of shadows' short film with prompt shared — LudovicCreator · 2026-11-19
- Stretto cuts voice cost 95% with an Amazon Nova 2 Sonic bankruptcy intake agent — AWS ML Blog · 2026-11-19
- Nynao pitches AI tool producing up to 400 finished video ads daily without editors — alifcoder · 2026-11-19
- Fable 5.2 iterates its own musical marble loop animation, fixing the hitch on second pass — dreamwieber · 2026-11-19