LoopVL: recurrent vision-language model beats larger non-recurrent peers, shows Visual Aha Moments

RUC · hf · 2026-10-01

RUC introduces LoopVL, extending Loop Transformers to vision-language models.

Combines Module-Loop and Model-Loop computation to iteratively update a unified vision-language state through shared modules, trained from scratch via language pretraining, multimodal training, and post-training.
Outperforms a range of similarly sized and larger non-recurrent models on multimodal understanding and visual reasoning benchmarks.
Observes "Visual Aha Moments": pronounced shifts in visual attention across loops.

The work offers practical evidence for recurrent vision-language modeling and a lens on how shared parameters support deeper multimodal computation.

More from Multimodal