Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels
Zhuchenyang Liu, Yao Zhang, Yu Xiao
cs.CV, cs.CL, cs.IR
2026-07-28
Having VLMs quote evidence instead of emitting bounding-box coordinates lifts evidence recall from a cap of 8 to 26-47 and roughly halves hallucination; a region-label-free GRPO recipe then raises an 8B model's strict attributed accuracy from 22.4 to 33.8.
Visual document QA has a strange failure mode: the model answers correctly but cannot point to where the evidence is in the document. The paper calls this Attribution Hallucination, a correct answer with a wrong or missing citation. The standard fix is to have the model emit bounding-box coordinates of evidence regions, but on a verified bilingual CiteVQA subset even a large model like Qwen3.5-27B barely reaches single-digit evidence recall (cap 8.1%), with hallucination rates of 82-97%.
The question the paper asks is sharp: is this a missing capability, or is the coordinate output interface itself the bottleneck?
They swap the coordinate interface for a language interface. The model no longer emits coordinates; it quotes the evidence verbatim (as a JSON answer plus a list of quotes). A layout parser (MinerU) segments pages into paragraphs, tables, and figure blocks; a multimodal encoder (Qwen3-VL-Embedding-2B) embeds both quotes and blocks, and a Hungarian assignment solves the linear matching that maps each quote to its page region. Tables and figures are quoted through their captions or notes.
The same quote-and-retrieve pipeline doubles as a training scaffold. Because region-level evidence labels are expensive for long documents, they design a GRPO recipe (with DAPO's asymmetric clipping) that needs no region labels at all. The reward comes from a vision-language judge (Qwen3.5-9B) scoring three dimensions (answer correctness, evidence relevance, evidence coverage), combined multiplicatively as r = (a/5) x (erel + ecov)/10. If the answer is wrong the evidence score is zeroed, so the model is forced to improve quotes only after it gets the answer right.
On a 719-question verified CiteVQA set, switching six open VLMs to the language interface lifts evidence recall from single digits to 26-47 and roughly halves the hallucination rate, with answer quality essentially unchanged (five of six models move within 2.3 points).
For Qwen3-VL-8B, recall goes from 0.3% to 39.9%, hallucination from 97.0% to 41.2%, and strict attributed accuracy (SAA) from 1.4% to 22.4%. Adding GRPO training pushes recall to 51.3%, hallucination to 28.4%, and SAA to 33.8.
| Interface | Recall | Hallucination | Strict attributed acc. |
| Coordinates (Qwen3-VL-8B) | 0.3% | 97.0% | 1.4% |
| Language (Qwen3-VL-8B) | 39.9% | 41.2% | 22.4% |
| Language + GRPO | 51.3% | 28.4% | 33.8% |
A credit-assignment ablation rules out "it just got coarser blocks." Snapping coordinates to layout blocks still caps recall at 23.5%; question-only retrieval underperforms quotes by 13-19 points; question-plus-answer retrieval closes the gap, showing the model encodes location into the language it generates. Multimodal encoding also beats lexical BM25 by 5-14 points. One honest ceiling: the layout parser itself tops out at 88.2% recall (IoU at least 0.5), which bounds the whole method.
The headline is a measurement warning. Much of what gets blamed on "the model cannot ground" is really the coordinate interface. Switch to quoting and the same models recover a lot. For anyone building document QA or RAG, attribution quality may not require retraining a big model; changing the interface captures much of the gain. The GRPO recipe shows you can train grounding without costly region labels.
The authors are explicit: only single-document attribution is covered; the model does not localize on its own (it depends on the parser and retriever, so the parser ceiling bounds results); purely graphical evidence without a caption cannot be quoted and is missed; and the assignment always returns a block for every quote, so hallucinated quotes are not flagged. Experimentally, the RL result is a single seed on one backbone, and evaluation is bounded by PDF availability (the verified set covers 72.9% of the original 987 questions).