Looped LLMs make contrastive decoding work like sequence-level guidance, researchers explain why

niloofar_mire · x · 2026-11-01

A blog by Xiaoyu Wu, Yifei Wang, Niloofar Mireshghallah, and Steven Wu explains why guidance/contrastive decoding works especially well in looped language models like Huginn and Ouro.

Looped models reuse the same block repeatedly to refine hidden states, beating non-looped peers on reasoning at equal parameter counts.
Key finding: predictions across loop depths are more similar than across layers in standard models, so the gap between early and final logits approximates sequence-level guidance.
Mechanism: extrapolate with zguided = zK + w(zK − zE), amplifying how token scores shift across loops.
The authors argue this ties LMs closer to diffusion-style guidance, making guidance a 'cheat code' for LLM reasoning.

Original post →

Looped LLMs make contrastive decoding work like sequence-level guidance, researchers explain why

More from Models