A Tree-based RAG Framework for Evidence-Intensive QA via Adaptive Planning and Topology-Aware Evidence Gathering
Songeun Lee, Kyungjin Min, Injae Na, Suyeong Lee, Chiyoung Kim, Woohwan Jung
EMNLP 2026
cs.AI, cs.IR
2026-09-04
APT-RAG grows a reasoning tree on the fly and gathers evidence along sibling and child links. On MoNaCo with Qwen3-30B it hits 50.84 F1 and 50.79 recall vs Plan*RAG 47.03 / 39.28.
Multi-hop QA usually needs a couple of pages (about 2.0 on HotpotQA). Evidence-intensive QA has to stitch fragments from dozens or hundreds: 43.3 gold pages on average in MoNaCo, 13.0 in QAMPARI. Structured RAG breaks at this scale in two ways.
Structural rigidity: methods such as PlanRAG freeze a graph up front, so one node may have to swallow a long entity list and a wrong filter poisons the branch. Topology-ignorant gathering: methods such as ToQ build a tree but collect evidence mainly at the root, so a child that already computed an attribute never reaches the parent. Baseline retrieval volume barely grows with gold-page count, so harder items stay under-covered.
APT-RAG grows a tree with depth-first recursion instead of drawing the full structure first. Each node has four steps.
Contextualization replaces <Qj> placeholders with finished sibling QA pairs, producing a self-contained question.
Adaptive planning: an answerability checker asks whether sibling QA pairs already suffice. If yes, lateral gathering fires and the node skips retrieval and decomposition. Otherwise a decomposer either rewrites the question into a retrieval query or splits it into children.
Topology-aware gathering has three modes. Lateral reuses sibling QA pairs and blocks repeated decomposition of already explored facts. External retrieves top-20 by cosine similarity. Vertical aggregates child QA pairs into the parent, turning a wide hunt into narrower ones. Each node reads only its own subtree and its siblings' subtrees; ancestor evidence is left to the ancestors, which avoids aggregating the whole tree at every node.
Evidence-guided batched answering cuts latency. Independent children assigned to external gathering are clustered by Jaccard overlap of retrieved passages and answered in one call. Clustering is a relaxed clique cover solved as coloring on the complement graph, plus a context-length cap. τS=0.0 (at least one shared passage); the window budget is 150k tokens for 4B and 50k for 30B.
Backbones are Qwen3-4B-Instruct and Qwen3-30B-A3B-Instruct, with Qwen3-Embedding-0.6B and k=20. MoNaCo main numbers use GPT-5.4 as judge. Baselines: LLM-only, NaiveRAG, PlanRAG, LogicRAG, ToQ. RT-RAG is too expensive for the full set and appears only on a 300-example subset in the appendix.
On MoNaCo with Qwen3-30B, APT-RAG reports 55.32/50.75/50.84 precision/recall/F1 and 50.79 retrieval recall. PlanRAG is 47.03 F1 and 39.28 recall. ToQ 46.66 / 31.91, NaiveRAG 46.64 / 31.68. LogicRAG has the highest precision (55.70) and the lowest retrieval recall (21.64), landing at 46.41 F1. The F1 lift over PlanRAG is about 8%; all F1 gains versus baselines are p≤0.001.
At 4B most baselines drop harder. PlanRAG's F1 is 27.02, below NaiveRAG's 34.98. APT-RAG still holds 40.69 F1 and 40.12 retrieval recall. On QAMPARI with 30B, F1 is 23.28 versus ToQ's 21.37, retrieval recall 19.61 versus 11.42.
As gold pages grow from 1–10 to 31+, baseline retrieval counts stay almost flat. APT-RAG's depth and breadth grow together, and Wikipedia pages retrieved rise with them. Answer F1 in the 21–30-page bin is close to what baselines score in the 11–20 bin.
Ablations split quality from speed. Removing vertical gathering collapses to NaiveRAG (F1 46.64). Breadth fixed at 2: F1 49.61, recall 37.48, 46.56s. Breadth fixed at the average 6: recall 51.60, 314.97s. The full system is 50.84 F1, 50.79 recall, 104.59s. Lateral gathering and evidence clustering cut latency 41.4% and 8.2%, with a larger drop at the 95th percentile. The cost is explicit: dropping both efficiency modules raises F1 to 53.96. The shipped recipe trades about 3 F1 for latency.
Coverage, not another layer of single-hop retrieval, is the bottleneck. The paper turns "harder question, bigger tree" into a working recursion and shows that frozen structures fall behind past 30 pages. Batched answering is an engineering patch, but tail latency is what decides whether this class of system can ship.
Spending 3 F1 for roughly half the latency is a product call: can the user wait a minute and a half.
The authors say inference is expensive and slow. The full system averages 104.59s; without the efficiency modules it is 184s. At 31+ gold pages, merged long context can eat the clustering savings, and mean latency gains reverse. Bad planning grows dead branches or forwards errors; the decomposer and answerability checker are still prompts, not trained modules. Only MoNaCo and QAMPARI appear in the main table. RT-RAG ran on a subset, so it is not aligned with the headline numbers.