Alibaba study finds most fancy contrastive tricks barely help multimodal embeddings; combos gain 6%
alibaba-multimodal-industrial-ai · hf · 2026-10-14
Alibaba Group and NTU systematically benchmark contrastive training strategies for multimodal embeddings on MMEB, finding most existing strategies offer only marginal gains over a standard aligned baseline — some even hurt performance.
- Introduces an MLLM-as-a-Judge diagnostic framework splitting top-1 retrieval errors into labeled positives, false negatives, and hard/easy negative errors
- Key findings: upweighting high-similarity negative gradients hurts generalization; wider positive-negative similarity separation doesn't imply better retrieval; combining hard-negative mining, false-negative mitigation, and hard-batch construction is optimal
- The combined recipe yields an average 6.0% improvement on MMEB with Qwen2-VL-2B
More from Research
- AIPOCH's Open-Science ships portable .science packages that share full research context, not just outputs — ahuja_priyank · 2026-10-14
- Video-Arena: Game-Based Video Benchmark Shows Native Models Score Just 13.2 F1 — Kuan Zhang · 2026-10-14
- LSCBench: Vibe Coding Agents Write Functional Code, But Only 25% Is Also Secure — Qwen · 2026-10-14
- Omni-LLMs Know More Than They Say: NAVER's OmniCatalyst Adds 7.29% via Self-Generated Text — naver-ai · 2026-10-14
- Repeated-Data Pretraining Study: Smaller Batches Act as Regularizers and Beat More GPUs — Sewon Min · 2026-10-14
- NAVER's WATCH: Cross-Video Human-Video Pretraining Lifts Robot Success Rate to 77.7% — naver-ai · 2026-10-14