Tsinghua & NUS Release Comprehensive Survey on LLM Memory Architectures
量子位 · wechat · 2026-08-05
A Tsinghua University team, collaborating with NUS and Bosch AI, has released a comprehensive survey on the memory architectures of Large Language Models (LLMs). The paper highlights that model memory is transitioning from a mere computational byproduct (like KV Cache) to a primary dimension in architectural design.
The researchers introduced a novel 3D architectural taxonomy for memory:
- Representation: Differentiating between compute-bound Implicit memory and independently addressable Explicit memory.
- Update Dynamics: Classifying mechanisms into Offline (training-only updates) and Online (dynamic inference-time updates, e.g., TTT).
- Persistence: Distinguishing between Short-term (session-bound) and Long-term (cross-session) memory.
The survey provides an in-depth analysis of the design philosophies behind Attention, SSMs, and MoEs, noting that Hybrid Architectures (like Kimi Linear and Qwen3-Next) are becoming mainstream, while also discussing future challenges in capacity, precision, and system costs.
More from Research
- Terence Tao on Human-AI Complementarity: AI Excavates, Humans Recognize — bennash · 2026-08-27
- Qwen3.8 Expert Analysis: 50% Can Be Pruned With Minimal Loss — EyalToledano · 2026-08-27
- Rogue Agents' self-naming habits spark interest in potential AI culture — DKokotajlo · 2026-08-27
- Paper: CoT Monitorability as a Fragile Safety Opportunity — idavidrein · 2026-08-27
- Modern LLMs Compress English Text to Under 1 Bit Per Character — docmilanfar · 2026-08-27
- LeakyLMs: Stealing Architecture and Inference Optimizations via Timing — niloofar_mire · 2026-08-27