RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection
Shicheng Xu, Liang Pang, Liyi Chen, Zihao Wei, Jingcheng Deng, Yan Gao, Yi Wu, Yao Hu, Huawei Shen, Xueqi Cheng
cs.CL, cs.AI
2026-08-03
RING learns retrieval internalization via a Mixture-of-Memory-Experts router trained with RL, beating SR-KI by 3.2 points on News-2025 while running 3-19x faster than RAG.
Retrieval-augmented generation keeps LLMs factual by grounding them in external documents, but it pays for that with inference-time latency and engineering overhead: every query has to hit a vector store, get its context assembled, then get passed to the model. Baking external knowledge directly into model weights instead, parametric injection, isn't a new idea, but existing approaches all have a specific weakness. Knowledge editing only handles a handful of isolated facts at a time. Continued pretraining forgets old knowledge as it learns new. Modular injection methods like KBLAM grow memory linearly with corpus size. Parametric RAG parameterizes documents but still needs an external retriever at inference to figure out which parameters to activate. All of them share a hidden assumption: retrieval itself can stay fixed and hand-designed, only the memory needs to be learned. RING asks whether retrieval can be learned too.
RING replaces a dense LLM's MLP layers with a sparse Mixture-of-Memory-Experts: a Basic Expert that keeps the original weights to avoid forgetting, a Knowledge Expert that parametrically memorizes new corpus content, and a learned router that picks between them at every token. Training runs in three stages. Continued pretraining injects the new corpus into the Knowledge Expert using a mechanism the paper calls Dual Causal Attention (DCA): standard causal pretraining only lets a token see what came before it, while DCA processes part of each target span with an added bidirectional view during training, giving the model richer context while it's memorizing, without changing the strictly autoregressive behavior at inference. Supervised fine-tuning then teaches a "search-then-answer" pattern in a single generative pass: the model first emits a retrieved memory fragment inside <retrieval> tags, then produces the final answer inside <answer> tags. Reinforcement learning (using GSPO) optimizes the router and Knowledge Expert with three reward signals, format correctness, how closely the retrieved fragment overlaps with the true supporting evidence, and whether the final answer is right, teaching the model to retrieve precisely without any external index at inference.
The paper builds News-2025, a benchmark of roughly 170,000 Chinese and English news articles published strictly after the base model's pretraining cutoff, so correct answers can't come from memorized pretraining knowledge or lucky reasoning. On a Qwen3-8B backbone, RING reaches 35.08% English and 32.36% Chinese accuracy, beating the strongest parametric baseline, SR-KI, by 3.23 and 3.14 points, and clearing every other injection method (KBLAM, LoRA/full fine-tuning, LAG, MLP Memory, AtlasKV). The margin over parametric baselines widens further on a Qwen3-14B backbone. Against external-search RAG, RING sits on the accuracy-latency frontier: on 8B, the strongest RAG pipeline (top-10 retrieval reranked to top-3 with Qwen3-Reranker) beats RING by 8.69 Chinese accuracy points, but its time-to-first-token is 10x higher; on 14B the tradeoff flips, RING beats that same pipeline on English and trails by under 2 points on Chinese while running 6x faster. Overall, RING is 3-19x faster than the RAG variants tested, with latency close to a plain fine-tuned model doing no retrieval computation at all. Ablations confirm each design piece earns its keep: sparse MoE alone adds about 2 points over a dense baseline, adding the learned gate adds another 2.4, DCA beats standard causal attention by 1.6 points, and the search-based reward beats an answer-only reward by nearly 3 points.
For high-QPS, latency-sensitive knowledge lookup, customer support, search suggestions, RING offers a path to keep most of RAG's accuracy while removing the external retrieval infrastructure entirely: no vector store to maintain, no retriever-generator mismatch to debug. The paper also frames RING theoretically as a discrete latent-variable approximation to the classical RAG objective, giving the router-plus-Knowledge-Expert design a principled argument, not just an empirical one, for why it can approach explicit top-K retrieval's expressiveness.
The authors are upfront about the biggest constraint: RING assumes a relatively stable knowledge corpus that can be injected offline before deployment; updating it requires retraining, which is far less flexible than an external RAG index that can be edited on the fly, making RING a poor fit for rapidly changing knowledge sources. Second, the generated <retrieval> block isn't a verbatim copy of the source document, the model can compress, paraphrase, or distort what it memorized, which complicates auditing and citation fidelity for applications like legal or compliance work that need strict traceability. Third, the evaluation is scoped to knowledge-intensive QA over a news corpus; it doesn't cover multi-hop web search, rapidly changing facts, multilingual knowledge bases, or long-form generation citing many sources, so whether RING generalizes to those settings remains open.