HKUST surveys in-parameter memory: storing post-training knowledge in LLM weights instead of context
HKUST · hf · 2026-10-07
HKUST researchers published a survey on in-parameter memory augmentation for LLMs and agents: encoding post-pretraining knowledge (domain facts, user preferences, documents, interaction experience) into parameters, adapters, or parameter-like objects composed into the forward pass at inference.
Motivation: in-context learning is flexible but consumes context capacity and incurs repeated encoding costs that grow with context length; parametric memory is a reusable complementary substrate.
Taxonomy (two orthogonal axes):
- Parameter Placement: Embedding, Attention, FFN layers, or Hybrid.
- Parameter Acquisition Time: online (acquired during deployment) vs offline (acquired before deployment).
The survey also covers open directions including interference, safety, co-design with ICL, and recursive self-improvement.
Related event: HKUST Survey Maps In-Parameter Memory Augmentation for LLMs(2 posts)→
More from Research
- New Hopfield model paper shows how memories generalize into hierarchical abstractions — ceciletamura · 2026-10-07
- LLM agents pick worse items from trusted sites; same price cuts Booking.com bias by 28 points — rohanpaul_ai · 2026-10-07
- Menlo Research open-sources Asimov 1 humanoid's locomotion policy and Isaac Lab training code — freelerobot · 2026-10-07
- Arm Introduces ALeWM, a JEPA World Model with Adaptive Latent Prefixes — Arm · 2026-10-07
- Microsoft's MiniCorp Simulates an AI-Run Company to Generate Enterprise Agent Data — microsoft · 2026-10-07
- NVIDIA's NeMo-DCR Cuts Trillion-Parameter RL Weight Sync from 87.5 min to 150s — nvidia · 2026-10-07