Looped-DiT: 260M Looped Model Beats 6.5x Larger Diffusion Models With 4.9x Less Compute
NandoDF · x · 2026-10-03
- New paper Looped Diffusion Transformer (Looped-DiT) scales computation by rerunning shared Transformer blocks within each denoising step, adding depth without adding parameters.
- Naive looping fails; the authors trace it to weak supervision on intermediate loops and unregulated attention updates that erode local information.
- Fixes: deep supervision across loops plus self-modulating attention to stabilize feature updates.
- A 260M-parameter looped model surpasses a 6.5x larger model on text-to-image benchmarks while using 4.9x less inference compute.
- Under fixed inference budgets, increasing loop depth yields larger gains than adding denoising steps, and deeper loops progressively correct earlier mistakes.
Related event: Looped-DiT: Recycled Transformer Layers Beat Models 6.5x Larger(4 posts)→
More from Research
- LEV: Looped decision model lifts 2.6B accuracy 63.2% to 76.2% with adaptive compute — jiqizhixin · 2026-11-19
- Aurora2-400M-A20M: a sparse MoE pretrained from scratch in 36 hours on one RTX Pro 6000 — Tall_Abrocoma_3533 · 2026-11-19
- Researcher calls METR/Anthropic '2x speedup' RSI analysis statistically dubious — jessi_cata · 2026-11-19
- Linear Probes Could Reveal Which Persona a Base Model Is Playing, Researchers Suggest — cephaloform · 2026-11-19
- ETH's Anymal robot completes 10km autonomous alpine hike with VLM-guided ROAM system — ChongZzZhang · 2026-11-19
- OpenVax codebase grows from 78K to 565K lines in 15 months — iskander · 2026-11-19