HOLA: Adding Hippocampus-like Precise Memory to Linear Attention
omarsar0 · x · 2026-07-03
A new paper proposes HOLA: adding a bounded exact KV memory module on top of the compressed recurrent states of linear attention or state space models, acting as a "hippocampal" supplement. Linear attention compresses the entire context into a fixed-size state to achieve O(1) memory, but early facts get overwritten and needle recall degrades when numerous key-value associations compete. HOLA restores long-term memory capabilities without sacrificing the efficiency of linear attention.
More from Research
- NUS builds a soft force sensor that drives actuators without electronics or power — CurieuxExplorer · 2026-07-27
- Chelsea Finn says robot RL is bottlenecked by physical rollout cost, not algorithms — ycombinator · 2026-07-27
- ICML 2026 oral paper replication scores stay middling after a stricter re-scoring — profjamesevans · 2026-07-27
- Long-running agents will need immutable event logs, this thread argues — sebpaquet · 2026-07-27
- Seed IQ navigates Doom II, prompting questions about benchmarks beyond ARC-AGI — Fit_Transition8824 · 2026-07-27
- Agentic Data Science in Practice: Agents Write Code but Answer Wrong Questions — hugobowne · 2026-07-27