KV cache vs recurrent memory: are tokens doing too much of the reasoning?

Public-Feedback-6705 · reddit · 2026-11-21

A Reddit deep-dive questions whether chain-of-thought reasoning should be carried entirely through token sequences when architectures with internal state updates exist.

Key points:

Core tension: should memory be retrieval from past tokens, or carried forward via internal connection changes? Recurrence risks losing details attention could retrieve, so the author argues for combining precise recall with continuously updated memory — especially relevant for agents on long, repetitive tasks. He asks practitioners whether deciding what to remember, retrieving it, or making recurrent compute competitive with attention is the harder problem.

Original post →

More from Research

Research channel →