KV cache vs recurrent memory: are tokens doing too much of the reasoning?
Public-Feedback-6705 · reddit · 2026-11-21
A Reddit deep-dive questions whether chain-of-thought reasoning should be carried entirely through token sequences when architectures with internal state updates exist.
Key points:
- Transformers use attention + KV cache for flexible recall, but the cache grows linearly with context
- Mamba-2 and RWKV compress sequences into selectively updated recurrent states
- Titans adds adaptive neural memory that learns what to retain
- BDH (Dragon Hatchling) takes a synaptic route: sparse interactions and connection updates as memory
Core tension: should memory be retrieval from past tokens, or carried forward via internal connection changes? Recurrence risks losing details attention could retrieve, so the author argues for combining precise recall with continuously updated memory — especially relevant for agents on long, repetitive tasks. He asks practitioners whether deciding what to remember, retrieving it, or making recurrent compute competitive with attention is the harder problem.
More from Research
- Genesis Molecular AI launches Ruby potency model, up to 1000x faster, open-sources training data — edunov · 2026-11-21
- 'All AI can do is plagiarize': model caught regurgitating a proof straight from The Book — moyix · 2026-11-21
- Dario says Anthropic found a new gene-editing mechanism; critics cite Feng Zhang's leaner record — max_paperclips · 2026-11-21
- GPTZero scans millions of citations: hallucination rates flat or rising at CVPR, ICLR, EMNLP — HanchungLee · 2026-11-21
- Custom RTS benchmark for agents: GPT-6.2 Sol scores 98.9, crushes field — asimpleabstraction · 2026-11-21
- Riemann Hypothesis by end of 2026: bold AI timeline prediction doubles down — dioscuri · 2026-11-21