Loopie Recurrent Transformer Outperforms Traditional Architectures at Equal Compute
Zitian Gao · hf · 2026-07-20
The Loopie series of recurrent Transformer models has been released, featuring two MoE versions: 20B total params (2B activated) and 6B total params (0.6B activated).
- Addressing Bottlenecks: Previously, increasing pre-training compute for recurrent Transformers was often less effective than simply scaling up parameters; Loopie overcomes this limitation.
- Performance: Ablation studies show that under an identical compute budget, its performance significantly exceeds traditional 30B-A3B baseline models.
- Reasoning Capabilities: Empowered by a novel post-training pipeline, the model achieves gold-medal-level performance in the 2025 IMO (International Mathematical Olympiad) and IPhO (International Physics Olympiad) without relying on external tools.
Related event: Loopie Looping Transformers Match Larger Models at Fraction of Cost(7 posts)→
More from Models
- Local beats cloud: Qwen 3.8 FN handles private student data as well as frontier models — dr_lm · 2026-10-27
- Aidan Gomez: model's CoT said no tool calls right before making them — aidangomez · 2026-10-27
- Nat Lambert: Cohere's model is narrow RL training on internal infra ill-suited for open weights — natolambert · 2026-10-27
- OpenAI teases DOTS is coming to Europe 'soon' — mark_k · 2026-10-27
- Clifford Pickover challenges AI to decipher a message he encoded in Norse runes — pickover · 2026-10-27
- ChatGPT's crawler sample code strips out all Schema markup — VeryWellVersed · 2026-10-27