Adobe's Chimera: Scaling Hybrid Visual Diffusion Transformers Efficiently
adobe · hf · 2026-07-31
Adobe introduces Chimera, a hybrid visual diffusion Transformer backbone designed to mitigate the quadratic cost of full attention in high-resolution image and long-video generation.
Architecture Highlights:
- Processes text, image, and video tokens in a single raster-ordered stream without positional embeddings.
- Combines Kimi Delta Attention (KDA) for O(N) long-context state tracking, interleaved Multi-head Latent Attention (MLA) for global interaction, and modality-aware short convolutions for local context.
- Integrates sparse Mixture-of-Experts (MoE) layers to expand capacity while controlling activated compute.
Scaling Laws & Results:
- Introduces HeterP, a module-wise hyperparameter transfer scheme, to fit Chinchilla-style compute-optimal laws for activated model size, training tokens, and data ratios.
- Trained an 11B parameter model (2B activated). The dense backbone is 1.7x as compute-efficient as a matched full-attention Wan-2.1 2B baseline, with the complete system reaching 7.3x efficiency.
- Demonstrates strong zero-shot extrapolation: generates 30-second videos from 5-second training clips with only 6.5% FID degradation in the last five seconds.
More from Multimodal
- Unreleased MiniMax H3 Surfaces with Native Multimodality and 12 Mixed Inputs — bennash · 2026-07-31
- H3-Omni Architecture: Native 2K Resolution and In-Context Regeneration — bdsqlsz · 2026-07-31
- Flux 3 and Minimax H3 Announce Open Weights for Video Generation — cocktailpeanut · 2026-07-31
- Runway Video Test: Generating a 15-Second Mumbai Monsoon Timelapse — CurieuxExplorer · 2026-07-31
- Testing Reference Image Generation with Hailuo-03 — chrisfirst · 2026-07-31
- MiniMax-H3 Tops Video Editing Leaderboard, Ties for 2nd in Text-to-Video — bdsqlsz · 2026-07-31