Researchers Debate Looping Only Middle Layers in Recurrent Transformers

Researchers proposed looping only the middle two-thirds of transformer layers in an encoder-loop-decoder design, arguing input and output mappings become dead weight after the first iteration, though critics note computation-graph depth, not layer count, is what matters.

2026-09-02 ~ 2026-09-03 · 3 related posts

• Why looping middle layers isn't enough: computational graph depth matters — _AndrewZhao · 2026-09-02
• Encoder-loop-decoder: a cleaner architecture proposed for looped transformers — letclaudiatweet · 2026-09-03
• Loop only the middle layers? Researchers debate looped transformer design choices — maxsloef · 2026-09-03