Deep Fusion Memory: chunk-level parametric memory cuts retrieval overhead in LLMs
Rubin Wei · hf · 2026-10-23
Shanghai Jiao Tong University's LUMIA Lab proposes Deep Fusion Memory (DFM), a chunk-level parametric memory architecture. A causal memory predictor generates memory from preceding context, injected into intermediate hidden states via gated cross-attention and supervised by corpus chunk embeddings. DFM avoids catastrophic forgetting of continued pretraining and online retrieval overhead, improving knowledge-intensive tasks while addressing the high cost of building parametric-memory training data and the shallow fusion of output-level interpolation.
More from Research
- Security researcher: AI-assisted research has 'given up on insight' entirely — mboehme_ · 2026-10-24
- arXiv: 'Can LLMs Anticipate Their Cruxes?' — a priori rationales fall short, rollouts required — rohanpaul_ai · 2026-10-24
- UChicago paper: LLMs can't anticipate their own cruxes — sampling many rollouts is unavoidable — rohanpaul_ai · 2026-10-24
- China's Jiuzhang 4.0 claims 10^42-year speedup in 25 microseconds — but the task has no practical use — AryHHAry · 2026-10-24
- Mila's EMNLP 2026 paper: thinking-images boost cross-view spatial reasoning — aagrawalAA · 2026-10-24
- Lightweight Formal Methods Could Save Vibe Coding: A 2016 OSDI Paper Gets a Second Life — tianyin_xu · 2026-10-24