EngramEdit: near-perfect LLM knowledge editing via DeepSeek Engram conditional memory
pmttyji · reddit · 2026-10-09
A new paper, EngramEdit, leverages conditional memory architectures like DeepSeek Engram — which look up learned embeddings via input n-grams — to decouple factual knowledge updates from the frozen Transformer backbone:
- Challenge: different phrasings of a fact activate different n-gram embeddings, and updating shared embeddings can break unrelated predictions
- Method: compute target memory representations that make the model predict the updated fact across multiple expressions, then jointly update shared n-gram embeddings with a stronger penalty on frequently reused ones to preserve unrelated knowledge
- Results: near-perfect editing success; updated knowledge generalizes to unseen expressions and multi-hop reasoning with 3x the strongest baseline's accuracy under CoT prompting; unrelated knowledge and general capabilities largely preserved even as updates accumulate
The upshot: conditional memory can serve as an editable knowledge interface, beyond just model scaling. Paper (arXiv 2610.10533), code, models, and project page are all public.
Related event: EngramEdit Paper Enables Decoupled Knowledge Editing via Conditional Memory(4 posts)→
More from Research
- Datology releases Zephon, a deterministic on-the-fly dataloader born from MosaicML Streaming's legacy — josh_wills · 2026-10-10
- Claude Code agents beat torch.compile on CUDA kernels: 2.5x fused GELU, 1.57x matmul via fp32-splitting trick — lmoroney · 2026-10-10
- ResLRB: constant-memory differentiable light tracing via stochastic graph compression — ssh4net · 2026-10-10
- Tsinghua's TokenRouter: Token-Level LLM Routing Hits Up to 64.15X Serving Throughput — rohanpaul_ai · 2026-10-10
- SpliceAI2 deep dive: splice graphs and DP reconstruct top transcripts for 82% of held-out genes — anshulkundaje · 2026-10-10
- OpenAI publishes 722 math manuscripts covering 372 open-problem results from internal model — FinanceYF5 · 2026-10-10