Meta's MoEMB: MoE scaling beats 4x-larger embedders with just 3B active params
_reachsumit · x · 2026-09-09
Meta researchers propose MoEMB, scaling universal multimodal embeddings along the expert axis with mixture-of-experts instead of larger dimensions or reasoning tokens. MoE preserves single-vector, non-autoregressive encoding while sidestepping the contrastive-learning batch-size tradeoff. With only 3B active parameters, MoEMB sets new SOTA on MMEB-V2 and MRMR among models trained on public MMEB-family data, surpassing reasoning-token (TTE) methods with >4x active parameters at far lower compute, plus the first comprehensive study of adaptive computation for MoE embedders.
Related event: Meta's MoEMB: 3B Activations Beat 4x Larger Embedding Models(2 posts)→
More from Multimodal
- Butter.video wins over skeptical creator with hundreds of tasteful timeline 'blocks' for video effects — beechinour · 2026-10-27
- AI Video Pits Brock Lesnar Against Duncan the Tall in a Toy Showdown — KloneStudios · 2026-10-27
- $50, 2-hour Kurzgesagt-style AI video reignites debate over intent in AI creation — motionbynick · 2026-10-27
- Fake BTS of a dam disaster made with Dreamina Seedance for 29 cents — Standard_Spell5873 · 2026-10-27
- Redditor struggles with outfit swap: Qwen Image 2.1 breaks body proportions — Existing-Box-7866 · 2026-10-27
- Kling 4.0 lands on Deevid AI: 30-second videos, 10 keyframes, 15 multimodal references — nikola_mr64990 · 2026-10-27