Hi-Q: Hierarchical Evidence-guided Query Refinement for Multi-Hop Question Answering
Jueun Kim, Sungho Park, Wook-Shin Han
cs.CL, cs.IR
2026-08-31
Hi-Q splits a query only when the reader cannot answer, in prerequisite-first order. Full-corpus average is 52.3 EM, 15.1 above IRCoT, without building a corpus graph.
A multi-hop question is often written at a coarser grain than the evidence the corpus can retrieve. Too coarse, and the retriever hits surface tokens such as "III" or "battle" without the full chain. Too fine, and constraints fall off. Graph RAG freezes corpus granularity before any question arrives; building PropRAG's graph over HotpotQA's 5.2M passages would cost more than $2,500 in API calls. Iterative rewriting, as in IRCoT, sees evidence but never checks whether the last query was already retrievable, so a bad bridge entity propagates.
POSTECH's Hi-Q treats multi-hop RAG as retrievable-granularity discovery. Each query node is tested against evidence and split only when it is not yet supported, in dependency order.
The state is the current sub-question, interaction history, and recursion depth. A resolution operator rewrites references from history, retrieves top-5 passages, and asks a reader to answer. A non-empty answer stops the node. Otherwise, if depth is below 4, a binary expander proposes a prerequisite branch and a dependent branch. The left branch is solved first; its answer rewrites the right. A coverage verifier checks that left-then-right still recovers the parent; one repair is allowed, then the node is marked non-decomposable. Synthesis uses consistency across branches rather than the single highest-scoring passage.
Stop versus expand is a cost-sensitive threshold: expanding a solvable node costs, and stopping on an unsupported one costs. In the implementation the reader's empty answer is the hard classifier. A query-only router is shown to have irreducible regret, because the same question can have opposite optimal actions under different evidence.
Retriever: NV-Embed-v2. Default reader: GPT-4o-mini at temperature 0.
Primary eval is full-corpus retrieval: 139k passages on MuSiQue, 430k on 2Wiki, 5.23M on HotpotQA, 1,000 questions each.
| Method | MuSiQue EM/F1 | 2Wiki EM/F1 | HotpotQA EM/F1 |
| PropRAG | 25.8 / 38.6 | 52.1 / 59.3 | — |
| IRCoT | 24.4 / 34.9 | 35.0 / 38.9 | 52.0 / 63.6 |
| Hi-Q | 37.4 / 50.6 | 62.0 / 71.4 | 57.4 / 69.9 |
Macro average is 52.3 EM / 64.0 F1, 15.1 / 18.2 above IRCoT. On the supporting/distractor pool used by prior graph-RAG work, Hi-Q averages 57.9 / 69.3, 5.6 / 3.9 above PropRAG. Dropping dependency order falls to 47.1 EM; a static one-shot split falls to 51.5; always-decompose is 3.4 EM worse than the failure-aware trigger on a 100-question slice.
When root retrieval fails, the leaf union at a matched top-5 budget lifts all-gold cover from 7.9% to 42.7% on clean MuSiQue triggers. Manual review of 100 triggers finds about 10% clear false fires. A cost-matched variant caps depth at 1, uses 2.93 LLM calls versus IRCoT's 2.92, 9.4× fewer input tokens, 8.6× lower API cost, and still +10.4 EM.
Multi-hop retrieval is often "retrieve more." Hi-Q treats granularity as a control: do not split a node that already answers, and resolve prerequisites first when you do. Gains are larger on full corpora than on pooled distractors, which is the setting that resembles deployment. The cost-matched point is both cheaper and more accurate than IRCoT; the full configuration spends about one third of IRCoT's token bill for another 3.3 EM.
No corpus-wide knowledge graph is required, which is the practical constraint at millions of passages.
Results are English Wikipedia, short-answer, acyclic dependencies. Mutually dependent premises and jointly optimized branches are outside the state. Reader refusals over-split; hallucinated answers stop too early. A binary resolved/unresolved bit cannot express partial or conflicting evidence. Trigger analysis is manual and small. Coding-agent baselines are controlled adaptations, not reproductions of published stacks. Long-form, multilingual, table, and multimodal QA are untested.