Knowledge Pull Requests for Continual Document Authoring
Alexander Martin, Benjamin Van Durme
cs.CL
2026-09-23
KPRs filter source claims and rewrite by section. Cross-lingual Wikipedia QA averages 69.2 vs 29.5 on the original; a 7B model with the revised article beats frontier search.
Wikipedia pages, technical docs, and intelligence reports are never finished. New sources, other languages, and later dates keep producing facts that belong in the standing document. The usual fixes are messy. Prompt a model with the old text plus new sources and you get a rewrite with no account of which claims were added or overwritten. Deep-research systems skip updating altogether and regenerate from scratch, throwing away the previous draft. Wikipedia itself tracks word diffs, not knowledge diffs. At millions of edits a month, reviewers cannot tell whether a change adds a fact, corrects a stale one, or fights existing content.
Software engineering already solved the analogous problem with the pull request: a proposal you can inspect, discuss, and merge. This paper moves that discipline onto documents as Knowledge Pull Requests (KPRs).
A KPR takes a main document and a set of sources. The pipeline has three stages, all run with Qwen3.5-27B:
The artifact is a ChangeLog: the claim proposal records what knowledge changes, the diff records how the wording changes.
Three baselines. ConText, adapted from WiNELL, rewrites each section from raw source text with no claim filtering. ConClaim conditions on source claims but skips coverage, conflict, and relevance filters. Scratch discards the standing document and regenerates from all sources; it is used only on RAGTIME. There is no human reviewer in the experiments, so flagged conflicts are withheld from the rewrite.
Two tasks. Revision: English Wikipedia is the main document and other-language editions from MegaWika 2.0 are sources, adding knowledge about an unchanged world. Update: three two-round variants of RAGTIME (temporal split, cross-round conflict, balanced nuggets) stand in for query-driven reports that must track new evidence.
Wikipedia quality is scored with MiRAGE. KPR reaches InfoP 0.878, retained recall InfoR-R 0.950, added recall InfoR-A 0.891. ConText is 0.837 / 0.946 / 0.716; ConClaim 0.853 / 0.943 / 0.786. Filtering claims is what lifts how much new information actually lands; retention is already high for every method.
As a grounding document for QA:
| Setting | Closed book | English original | ConText | ConClaim | KPR |
| En-QA average | 35.8 | 92.0 | 87.6 | 88.0 | 87.7 |
| Multi-QA average | 36.9 | 29.5 | 48.4 | 58.5 | 69.2 |
On English questions the three rewrites drop a little versus the original and sit within half a point of each other. On cross-lingual questions the untouched English article falls below closed book, because the facts are simply not on the page; KPR lifts the average to 69.2.
On the hardest 100 multilingual items (no dense open model answers them closed-book), Sol-5.6 with web search scores 38, and 41 if the question is translated into the source language first. The same model grounded on the KPR article scores 62. OLMo-3-7B on that article scores 50, already above the frontier model with search. The knowledge is public, indexed other-language Wikipedia. Search still does not surface it.
Review cost: KPR has 11.2 contiguous edit blocks, keeps 97.3% of the original, and expands to about 2.3×. ConClaim emits fewer tokens but 25.3 clicks, so the reviewer faces more fragmented edits.
Across all three RAGTIME settings KPR leads on both retained and added recall. In the conflict setting ConText precision falls to 0.666; KPR withholds conflicting claims and holds precision 0.811 with added recall 0.571, instead of resolving disagreements inside the rewrite. Scratch is precise and stingy: added recall is only 0.223 in the temporal split, and only about 37% of the original text survives.
Claims were already the right unit for factuality scoring. This paper uses them for continual authoring: filter first, then rewrite. More of the new knowledge lands, more of the old document stays, and the edits clump into blocks a person can approve. That is especially useful for a local 7B model: pair it with a KPR-revised article and it recovers knowledge that a frontier model with search still misses. Deep-research-style regeneration is the expensive, low-yield option here.
Decomposition, classification, routing, and rewriting are all LLM calls. The edit-cost metrics measure the finished diff, not the compute spent producing it; continually updating Wikipedia this way would be expensive, and the authors note that a light encoder could replace classification and routing. There is no human-review study, so it is unknown whether a ChangeLog makes an editor faster or more accurate than a text diff. Withholding conflicts ducks the question of which source to trust. RAGTIME section structure is imposed after the fact, not native to the reports.