WeMM-Embedding tops MMEB-v2: its 2B model edges out 8B rivals at a quarter of the size
tomaarsen · x · 2026-09-02
WeMM-Embedding's average scores on MMEB-v2 (78 datasets):
8B/9B class
- WeMM-Embedding-9B: 80.6
- DME-Medium (closed): 78.4
- Qwen3-VL-Embedding-8B: 77.8
2B class
- WeMM-Embedding-2B: 77.9
- Qwen3-VL-Embedding-2B: 73.2
The key takeaway: WeMM's 2B model (77.9) edges out Qwen3-VL's 8B model (77.8) at roughly a quarter of the size, while the 9B tops the closed-source DME-Medium on the overall leaderboard.
More from Models
- Open-weight TYGER speech decision model makes 80ms turn-taking calls for full-duplex voice agents — solyarisoftware · 2026-10-18
- Paying Google AI subscriber blocked: custom-domain Workspace accounts can't access AI Studio — ChrisRackauckas · 2026-10-18
- Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC — BerryBlush_Rogers691 · 2026-10-18
- Users press OpenAI on long-delayed Adult Mode a year after it was promised — ___Patrice___ · 2026-10-18
- Polymarket opens betting market on who releases the next Gemini Argon model — Polymarket · 2026-10-18
- Grok matter-of-factly lists which groups user was racist against, exempting Africans and Latin Americans — teortaxesTex · 2026-10-18