Chaos in the Text: Mixed-Modality Retrievers Favor Text, Letting Irrelevant Text Outrank Relevant Images
_reachsumit · x · 2026-10-09
Paper [2610.11816] systematically examines dense retrievers on mixed-modality corpora.
- Key finding: as image documents are replaced with semantically corresponding text, retrieval performance follows a pronounced V-shaped curve—strong on single-modality corpora, degrading substantially when modalities coexist
- "Chaos in the Text": irrelevant text causes worse degradation than an equal number of irrelevant images; text representations receive systematically higher similarity scores, letting irrelevant text outrank relevant images
- Fix—Trident: builds text, image, and fused views of each document as co-equal positives, jointly optimized via Multi-Positive View InfoNCE
- Improves mixed-modality retrieval on both CLIP-based and VLM-based architectures
Related event: 'Text Chaos': Irrelevant Text Hurts Multimodal Retrieval More Than Images(2 posts)→
More from Multimodal
- AI-generated movie poster 'The Chase' crams a wolf, a plane and muscle cars into one chase — Aggravating-Sir-115 · 2026-10-23
- SteerTTS-edge: an 11.84MB steerable TTS with unchanged WER running 19x realtime on CPU — Working_Hat5120 · 2026-10-23
- ElevenLabs' Chimera pairs Eleven v4 voice with Seedance video to make character images speak — AIwithGhotai · 2026-10-23
- Open-Source Tennis Models Plus Claude Turn Video Into a World-Class Coach — tweetsatpreet · 2026-10-23
- Berkeley's StyleStream 2.0 cuts streaming voice conversion latency to 520ms, 2x faster — Berkeley · 2026-10-23
- AuroraTTS-100K: a sub-100K-parameter English TTS running 1120x real-time on GPU — Tall_Abrocoma_3533 · 2026-10-23