InfiLoop residuals let a 7M looped model improve past 20,000 test-time steps, 97.9% on Sudoku

Pengxiang Li · hf · 2026-10-09

• Problem: more loop iterations can hurt looped Transformers—noisy state updates overwrite correct intermediate deductions, and early-loop errors propagate through recurrence.
• Fix: InfiLoop, a loop-native residual connection that learns which past computations to retain and how much to accept from each update, using content-based weighting plus learned temporal decay with an exact streaming recurrence of constant memory.
• Results: a 7M-parameter model beats existing recursive architectures—97.9% exact accuracy on Sudoku-Extreme, 13.6% pass@2 on ARC-AGI-2—and keeps improving past 20,000 effective steps. Code: github.com/pixeli99/InfiLoop.

Original post →

More from Models

• Enjoying the Centaur Era While It Lasts: OpenAI's Math Results Signal Its End — moyix · 2026-10-09
• Agent Arena Leaderboard: Claude Opus 5.5 Tops GPT 6 Astra Across 2.3M Agent Sessions — arena · 2026-10-09
• Same prompt without thinking mode: a stark model quality gap — umesh_ai · 2026-10-09
• Auditing 162 LLM benchmark gaps: only 20 survive the noise check — maverick_man1111 · 2026-10-09
• Qwen3.8-Max, Flash and Wan3.0 free for a week on GMI Cloud with higher rate limits — Alibaba_Qwen · 2026-10-09
• ByteDance Seed team spotted DeepSeek performance drift, Zhihu explainer goes viral — teortaxesTex · 2026-10-09

Models channel →