RAG Deserves an Index: Why Ingest-Time Compilation Beats Query-Time Interpretation
Kyle Wild, Yusuke Takahashi, Asako Uraki
cs.AI, cs.DB, cs.IR
2026-08-21
ISC compiles a corpus into provenance-checked claims at ingest. On 500 transcripts, claims reach 85.2% at 2.2k tokens versus 72.5% at 16.3k for chunks.
Production RAG pays a hidden interpreter on every query: a language model re-resolves references, attribution, and structure in raw text, then throws the work away. Per-token prices fell by orders of magnitude while aggregate inference spend rose, because context volume grew faster than prices fell. Endgame Labs and Musashino University call this query-time semantic reconstruction (QSR) and analogize it to a full-table scan. The proposed fix is the one databases found fifty years ago: do the expensive work once at write time, into a maintained structure.
That structure is ingest-time semantic compilation (ISC): compile a corpus's meaning into a queryable substrate and treat the substrate as a first-class database object.
The substrate has two coupled layers. The geometric layer is incrementally maintained embeddings, answering "where is meaning like this?" The symbolic layer is atomic claims, each bound to a verbatim span, speaker, and location. A compile-time exact-quote gate drops candidates that fail; nothing is patched through. The reader receives the claim plus its evidence span, not a raw window it still has to interpret.
Four contracts govern the object: a compilation contract (semantic DDL: no exact quote, no row), a maintenance contract (cost tracks change, not corpus size), a migration contract (an embedding-model swap is an alignment problem, not a rebuild), and a cost model (compile when expected reads exceed a break-even count R). The reference implementation is ordinary PostgreSQL: a facts table joined to factevidence, with UNIQUE(documentid, contenthash) making recompilation idempotent.
The maintenance study is synthetic: an embedding corpus grown from 3,000 to 9,000 documents over 50 updates. Incremental low-rank updates cost 8.4 ms versus 283 ms for full re-decomposition, 33.7× cheaper per update and 23.8× cheaper cumulatively, with principal-angle drift below 10^{-11} degrees and recall@10 of 1.0. An embedding-model swap recovered 0.95 mean cosine to truly re-embedded vectors by re-embedding about 10% of the corpus via orthogonal Procrustes. The authors flag these as best-case bounds, not yet measured on production embedding APIs.
The read-path study uses MediaSum broadcast interviews, with 500 held-out transcripts and 499 questions sharing no document with development. Compiled claims as the retrieval payload win all 32 budget-by-model cells against chunking: 85.2% correct from about 2.2k reader tokens at a 2,048-token budget, versus 72.5% from about 16.3k for the best chunk cell. Facts at 256 tokens (69.5%) beat every chunk policy at 2,048. The strongest chunk stack (LLM contextualization, hybrid retrieval, cross-encoder rerank) reaches 88.0% (439/499) at 2,048, statistically tied with claims at 85.2% (425/499), McNemar p=0.202, at about 47.7k query-path tokens, roughly 21× more.
Compilation has a price. Turn-wise extraction with Kimi K2.6 averaged 48.2k prompt and 4.5k completion tokens per document, about $0.064, or about $32 and 26.3M tokens for 500 documents. Against the extra 45.5k query-path tokens of the full stack, compiling the corpus is about 580 of those queries. In a 20-document replay the gate rejected 1.1% of candidates (29 of 2,724), 28 of them because the quote could not be located in the source.
The split that matters is payload versus pointer. Doc2Query, proposition indexing, and contextualized chunking mostly compile the pointer; the reader still consumes raw text. ISC compiles the payload and treats exact quotes as an integrity constraint. For meeting notes, interviews, and internal corpora that are asked repeatedly, this looks more like a materialized view than another chunking trick. A small, highly volatile, or rarely queried corpus may never reach R; query-time interpretation stays the right call. The paper treats "compile or not" as a planning decision with a number, not a doctrine.
Maintenance numbers come from synthetic vectors. Streaming revisions, crash recovery, and a semantic redo log remain agenda items. Answer grading and extraction share Kimi K2.6; human calibration and a re-grade under another model family are still in progress, so the read-path result is labeled preliminary. When a tool-using agent mediated retrieval, the compiled-payload advantage did not automatically survive. R≈580 is a token equivalence, not a dollar one, because compile-time and query-time models are billed differently. Dialogue is the measured domain; long documents or multi-hop structural queries could rewrite the frontier.