VecTree-RAG: An Agentic Retrieval-Augmented Generation Framework Combining Vector and Tree Retrieval for Efficiency and Accuracy
Xinyan Zhong, Yuwei Shi, Yuqi Wei, Chen Shen, Tianhang Zhou, Zhenghao Wu
cs.IR, cond-mat.mtrl-sci, cs.AI
2026-07-25
VecTree-RAG uses vector search to pick papers and section-tree traversal to find evidence inside them, topping QASPER, LitQA2 and MOSAIC, with evidence-page precision 0.274 vs 0.046 to 0.071 for baselines.
Scientific question answering has two layers: first find which papers in a corpus are relevant, then locate the supporting evidence inside those papers. Mainstream RAG handles both by chunking every document into fixed-length passages and running similarity search over all of them.
Chunking throws away the section structure of a paper. A method statement cut into a passage gets separated from the experimental setup, figures and caveats it depends on. More retrieved passages just means more redundancy; the thing readers actually want, which page the claim sits on and which method backs it, gets harder to pin down.
VecTree-RAG assigns the two jobs to complementary mechanisms and follows a locate, navigate, read pipeline that exposes content progressively.
The vector layer does cross-document filtering. A hybrid of SPECTER2 dense vectors and BM25 sparse retrieval scores compact representations of each document and section, with relevance gates (dense similarity at least 0.35 or BM25 at least 1.0) to shortlist candidate papers.
The tree layer handles navigation inside a document. Each paper is parsed into its native section hierarchy as a section tree, where every node carries a title, summary and the page range it covers. Crucially, page numbers extracted by an LLM are verified by deterministic substring matching within a one-page tolerance, to avoid hallucinated coordinates. Full text lives in a page store at page granularity; pages are fetched only after a node is localized.
Execution is a ReAct-style agent with seven tools across the three layers, capped at 30 iterations. The system prompt hard-codes two rules: read section-tree summaries before requesting pages, and never fetch more than five pages without synthesizing an answer.
VecTree-RAG posts the highest answer score on all three benchmarks:
| Benchmark (size) | VecTree-RAG | Best baseline |
| QASPER (300 q, LLM-judge correctness) | 0.800 | 0.757 (reranked Dense RAG / Search-o1) |
| LitQA2 (54 q, accuracy) | 0.925 | 0.889 (reranked Dense RAG) |
| MOSAIC (49 q, composite) | 0.547 | 0.503 (Dense RAG) |
The widest gap is in evidence localization. On QASPER, evidence-page precision is 0.274 versus 0.046 to 0.071 for the baselines, roughly a 4x to 6x difference. It does not just answer correctly; it points back to the exact page.
The cost is tokens. On LitQA2 it spends about 128k tokens per question, against 9.7k for Dense RAG. The efficiency the authors stress is about index representation and navigation, not end-to-end inference, since multi-turn agent calls are inherently pricier than single-shot retrieval.
The ablations are telling: drop the tree layer and accuracy falls to 0.904; drop the vector layer and accuracy falls to 0.811 while tokens balloon to 490k per question. Both layers are needed, but the vector layer is where the money is saved.
For scientific-QA settings that need traceable evidence (research assistants, compliance review, medical QA), this is a more honest direction than retrieving more passages: respect document structure, localize first, then read closely. The evidence-page precision metric directly attacks the old RAG habit of being right without being able to say where it came from.
It is not cheap; the multi-turn agent cost is real. It fits scenarios where answer quality matters more than per-query latency, not lightweight Q&A chasing low latency.
The authors flag a long list. The method assumes informative section structure and may transfer poorly to unstructured documents. Section summaries are LLM-generated and their faithfulness is not currently verified. Evidence localization stops at paragraph or page granularity, not sentence level. MOSAIC, the multi-document benchmark, was generated and screened automatically without independent human validation, and its construction and scoring use the same model family (DeepSeek), a self-evaluation risk. Finally, the efficiency claim covers only indexing and navigation; end-to-end tokens are actually higher than baselines, so do not overread the word efficiency in the abstract.