Loopback Transformer Matches Standard Transformers with 40-75% of Parameters
A new paper proposes the Loopback Transformer, which reuses shared modules to cut parameter counts. It matches or exceeds standard Transformers, including a 3B model, using only 40-75% of the parameters.
2026-11-22 ~ 2026-11-23 · 2 related posts
- Loopback Transformer beats standard baselines with 40-75% fewer Transformer parameters — Crazy-Repeat-2006 · 2026-11-22
- Loopback Transformer matches 3B model with 75% of parameters, gains mostly from decay — pbaylies · 2026-11-23