Looped layers don't help multimodal embeddings unless recurrence is pretrained (+4.23 points)

_reachsumit · x · 2026-11-18

An arXiv study tests whether running the same layers again improves multimodal embeddings. Across 11 loop designs, 3 checkpoints, and all 78 MMEB-V2 datasets, adding a loop to a standard model loses 0.46 points vs its control; adding an intermediate generative stage flips it to +0.36. Only a model pretrained with recurrence gains (+4.23 points over its loop-removed control). The authors prove query-independent image states persist under forward-only information flow and show query-conditioned visual updates matter mainly for visual Q&A.

Original post →

More from Multimodal

Multimodal channel →