Meta FAIR Trains 200+ Models to Show Vision Prior Is Optional but Vision Capacity Isn't
mengyer · x · 2026-11-14
A Meta FAIR/NYU paper (arXiv:2611.17334) disentangles the three choices bundled in vision encoders via controlled pretraining of 200+ models on shared data and compute. Findings: the upstream visual prior is unnecessary — an encoder trained jointly from scratch matches a pretrained one without a separate stage; dedicated vision capacity is what matters — encoder-free models trail even when the backbone is enlarged to compensate; and that capacity need not precede fusion — in-fusion vision modules in early backbone layers perform comparably and cut time-to-first-token. This reconciles diverging lab recipes (Gemma 4 dropping the encoder, Kimi K3 skipping encoder pretraining).
More from Multimodal
- OpenAI Images 2.6 tops DesignArena and Image Arena at 4s per image, same price — thursdai_pod · 2026-11-14
- MiniMax H3 Camera & Motion Control Tested on a RTX 4070 with Only 8GB VRAM — big-boss_97 · 2026-11-14
- 3D scene gen comparison: Inkling 2 Small cinematic, Kimi K3 bland — LiTianleli · 2026-11-14
- Anthology episode '2000' made entirely with AI tool Cuelist — Uncanny_Harry · 2026-11-14
- BudgetAny family extends BudgetPix to image-to-3D with one checkpoint, multiple inference budgets — anand_bhattad · 2026-11-14
- Google Flow's Omni 1.2 Flash Makes a Full 15-Second Beverage Ad from One Prompt — Emergency_Abies_6508 · 2026-11-14