Meta's FLAT unifies image-text representations with 1D flexible-length tokens, hits 83.1 GenEval
meta · hf · 2026-09-17
Meta introduced FLAT (Flexible-Length Aligned Transmodal representations), a pre-training framework that jointly trains a shared multimodal encoder with text-to-image and image-to-text decoders, replacing the traditional two-stage setup where generative performance is bottlenecked by frozen embeddings.
Key ideas
- Combines contrastive alignment with bidirectional cross-modal generative objectives in a single pre-training stage, so representations serve as both discriminative descriptors and generative conditions.
- Maps visual and text inputs into a unified continuous 1D sequence space, using nested dropout over prefix-K tokens to enable dynamic output lengths.
Results
- Zero-shot T2I generation: 71.1 GenEval
- After task-specific fine-tuning: 83.1 GenEval on T2I; 40.5 BLEU-4 / 138.6 CIDEr on MS-COCO captioning; Recall@5 of 86.8 (I2T) / 75.8 (T2I) on MS-COCO and 98.3 / 93.6 on Flickr30k
Qualitative evaluations show FLAT embeddings natively support linear interpolation, latent-space arithmetic, and zero-shot composed retrieval.
More from Multimodal
- HunyuanVideo 1.5 Outputs Black Video: Only Certain Frame Lengths Render — Big-Maybe-2228 · 2026-09-17
- QuiverAI launches Arrow 2, its most advanced model for editable vector graphics — stuffyokodraws · 2026-09-17
- SuperSplat now shows splat counts for your 3D scenes — willeastcott · 2026-09-17
- ai-toolkit, the open-source LoRA trainer, lands on Pinokio for 1-click installs — cocktailpeanut · 2026-09-17
- Spotify's AI Persona Badge Targets Photorealistic Identities, Not Royalties — mixtapedmonk · 2026-09-17
- Blender + AI modeling tip: design parts separately and keep everything assemblable — kevinkern · 2026-09-17