Leveraging External Knowledge for Historical Document Restoration via Retrieval-Augmented Large Language Models
Gabeen Kim, Kyeongpil Kang
ACL 2026 Findings of the As
cs.CL
2026-07-24
Nearly half the damaged characters in historical records are named entities that local context cannot recover; ARI injects retrieved historical corpus into an LLM via RAG, lifting named-entity restoration from 5.57% to 27.85% and topping expert review with a 46% win rate.
Korea's Journal of the Royal Secretariat (JRS) has about 41.9K damaged characters across 11.1K documents. Of those damaged characters, 44.8% are named entities: people, places, official titles, era names.
The problem is that existing restoration is basically masked language modeling, which guesses a blank from the surrounding context. That works for ordinary characters but fails for named entities, which often need historical knowledge beyond the document. The authors' English example: filling [M] in "In 1492, Columbus first landed in [M]" requires knowing the history; the sentence structure alone is not enough.
The framework, ARI (Archive Restoration Intelligence), combines three things: the implicit historical knowledge of a pretrained LLM (Qwen3 32B/8B), explicit external corpus retrieved by RAG, and fine-tuning on classical Chinese-character (Hanja) text.
The input is structured: a task description, temporal metadata (reigning king, year, month, day), retrieved related documents, and the damaged text with positional markers [Dn]. The retrieval is tuned carefully. BM25 beats embedding-based and random retrieval; they retrieve 20 documents and deduplicate at an 80% similarity threshold. The baseline is BERT-Res, a ModernBERT-large trained from scratch with an extended Hanja vocabulary and 25% of masks prioritized on named entities.
The ablation on JRS shows each component's contribution clearly (Qwen3-32B named-entity accuracy): baseline prompt 5.57%, adding metadata barely helps (5.63%), static examples nudge to 6.27%, dynamic BM25 examples jump to 24.28%, and deduplication reaches 27.85%. RAG plus deduplication accounts for almost all the gain. On the same setting, random-mask character accuracy rises from 13.37% to 61.45%.
| Configuration (Qwen3-32B) | Named-entity acc. | Random char acc. |
| Baseline prompt | 5.57% | 13.37% |
| dynamic examples (BM25) | 24.28% | 60.36% |
| deduplication (80%) | 27.85% | 61.45% |
Against proprietary models without RAG, Sonnet 4.5 leads on named entities (13.30%), ahead of Gemini-2.5-Pro (9.57%) and Kimi K2 (6.53%). Fine-tuned ARI-32B tops both named-entity and random-character accuracy, beating much larger closed models.
The most persuasive result is expert review. Three experts in Sinographic literature and Korean history scored 100 real damaged documents: ARI-32B reaches 38.3% accuracy@1 and a 46% win rate, ahead of Gemini-2.5-Pro (27.3% / 30%) and BERT-Res (20.7% / 24%), with nDCG@10 of 0.477.
This is a textbook old-task-new-tool paper. Document restoration has been dominated by MLM, and RAG has rarely been applied systematically. The transferable lesson is that any fill-in-the-blank task needing external domain knowledge can capture most of its gain from retrieval plus an LLM, and you do not need the largest closed model. A 32B open model fine-tuned can surpass them.
For digital humanities and archives, it is a usable tool, and the code is open.
The authors list several: a possible distribution gap between the constructed dataset and real damaged documents; a 4,096-token input limit that filters about 1.7% of longer data; text-only, with no visual features from the original document images; and performance degradation on temporally distant corpora (such as the Goryeosa, predating the training set) due to diachronic language change. Training cost is real: about 1,500 H200 GPU-hours.
One concern on reading: expert review covers only 100 documents and three experts, a small sample, and the RAG gain depends heavily on retrieval quality. The drop on unseen temporal domains shows the method is sensitive to whether the retrieval corpus actually contains relevant material.