DeepMind's ELT: Elastic Looped Transformers Cut Visual Generation Parameters by 75%

adityakusupati · x · 2026-08-08

Google DeepMind introduces ELT (Elastic Looped Transformers), a highly parameter-efficient model for image and video generation.

Core Mechanism: Unlike conventional deep stacks of unique layers, ELT employs iterative, weight-shared transformer blocks to drastically reduce parameters. It is trained via Intra-Loop Self Distillation (ILSD) to ensure consistency across depth.
Elastic Inference: Yields a family of elastic models from a single training run, enabling Any-Time inference with dynamic trade-offs between compute cost and quality.
Performance: Achieves a competitive FID of 2.0 on class-conditional ImageNet 256x256 and FVD of 72.8 on UCF-101, with a 4x reduction in parameters under iso-inference-compute settings.
Training Speedup: New experiments show that freeing up HBM via looped architectures allows trading memory back for speed (e.g., using unrolling/for-loops instead of jax.lax.scan).

The paper has been accepted to ECCV 2026.

More from Multimodal