Recurrent Looped Transformer feeds final hidden states back for deeper reasoning
WebAssemblyMan · reddit · 2026-09-16
Recurrent Looped Transformer (RLT) passes the decoder's final hidden state to the next token alongside its causal encoder representation, reads global KV memory, and maintains a sliding-window attention cache at every layer — the same update runs over prompt and response tokens. The claimed benefit is more effective reasoning depth; code is on GitHub.
More from Research
- CoLLAs 2026: Joseph Campbell argues introspection is a core mechanism for lifelong agents — apsarathchandar · 2026-09-16
- Teaching a robotic hand to walk on its fingertips, no robot arm required — Scobleizer · 2026-09-16
- After searching thousands of learning rules, none beat backprop—and Nature confirms it's biologically plausible — aran_nayebi · 2026-09-16
- BoltzMol-1 finds WRN inhibitor hits for ~$10K in 10 days; best IC50 8.4 µM — GabriCorso · 2026-09-16
- Researchers clash over Sakana AI's bioplausible learning claim: MNIST results don't count — aran_nayebi · 2026-09-16
- A 0.62-AUC model made a 65,578-candidate materials search tractable, doubling hit rate — bravo_abad · 2026-09-16