Meta's FLAT unifies image-text representations with 1D flexible-length tokens, hits 83.1 GenEval
meta · hf · 2026-09-17
Meta introduced FLAT (Flexible-Length Aligned Transmodal representations), a pre-training framework that jointly trains a shared multimodal encoder with text-to-image and image-to-text decoders, replacing the traditional two-stage setup where generative performance is bottlenecked by frozen embeddings.
Key ideas
- Combines contrastive alignment with bidirectional cross-modal generative objectives in a single pre-training stage, so representations serve as both discriminative descriptors and generative conditions.
- Maps visual and text inputs into a unified continuous 1D sequence space, using nested dropout over prefix-K tokens to enable dynamic output lengths.
Results
- Zero-shot T2I generation: 71.1 GenEval
- After task-specific fine-tuning: 83.1 GenEval on T2I; 40.5 BLEU-4 / 138.6 CIDEr on MS-COCO captioning; Recall@5 of 86.8 (I2T) / 75.8 (T2I) on MS-COCO and 98.3 / 93.6 on Flickr30k
Qualitative evaluations show FLAT embeddings natively support linear interpolation, latent-space arithmetic, and zero-shot composed retrieval.
More from Multimodal
- Midjourney wallpaper recipe: sref + profile mix for eerie film-blur portraits — michaelrabone · 2026-11-02
- Cavallaero Teaser: A Painterly Rider Races Through a Golden Forest — CommunicationTop4244 · 2026-11-02
- Anima's snapref Offers a New Reference-Based Fix for Character Consistency — Intrepid_Swim_8669 · 2026-11-02
- Pencil-textured folding fan: surreal product concept render goes viral — Delahuntagram · 2026-11-02
- Free Midjourney SREF gallery adds 1,200+ handpicked style codes overnight — tisch_eins · 2026-11-02
- Adobe's EditAgent beats prior image-editing agents but needs ~10x the compute — emmanuelvivier · 2026-11-02