Query Expansion Is More Than Generation: Improving Dense Retrieval through Better Integration
Siyuan Sun, Mihai Surdeanu
cs.IR
2026-08-26
Holding expansions fixed, AnchorQE interpolates query and expansion vectors. It beats published concatenation by up to 12.89% on TREC-DL, LoTTE, and BEIR.
Zero-shot LLM query expansion should help a frozen dense retriever. Often it does not. Sun and Surdeanu freeze the generated text and change only how that text enters the encoder. Expansion-only search (HyDE) and query-expansion concatenation (Query2Doc, prompted terms) fall below the no-expansion dense baseline on most of TREC-DL, LoTTE, and BEIR, averaged over four generation strategies. Generation quality cannot explain a gap that appears when the text is held fixed. Once query and expansion share a token sequence, tokenization, attention, pooling, and truncation entangle them, with no knob for how far the expansion may drag the query vector.
AnchorQE encodes the original query and the expansion separately, L2-normalizes each, and interpolates: qα = normalize((1-α)q + α m). α=0 recovers the query. α is the total expansion weight. Multiple expansions mix into m first; disagreement shrinks ‖m‖ and weakens the correction. In a shared dense space with raw dot-product scores, one-vector retrieval ranks identically to weighted CombSUM while issuing a single index request. For α<1/2 the method also has an angle bound: at α=0.10 no expansion can rotate the query by more than 6.38 degrees.
SC-AnchorQE estimates α from an unlabeled prefix, eight queries by default. Each prefix query is probed once with q and once with z. Two ratios are formed: relative top-1 retrieval strength of the expansion, and agreement of the expansion with the original query's top-10 evidence. Their product is α, then frozen. Later queries make one AnchorQE request. Trust is high only when the expansion both retrieves strongly and agrees with the query's own evidence.
The controlled comparison saves HyDE, Query2Doc, Q2E keywords, and CoT terms from Qwen3-8B once, and keeps BGE-large-en-v1.5 plus the document index frozen. A stronger text baseline picks among 72 concatenation recipes on MS MARCO dev; fixed α=0.15 is chosen on the same split.
In the diagnostic table, published integration and tuned text re-encoding each fall below the dense baseline in 16 of 20 strategy-benchmark cells. AnchorQE at α=0.15 beats both the baseline and the published recipe in all 20. Relative gains versus the published recipe run from 0.46% for Query2Doc on TREC-DL 2019 to 12.89% for CoT terms on BEIR-14.
SC-AnchorQE beats published integration in all 20 cells. The largest relative lift is 13.03% for CoT on BEIR-14 (nDCG@10 0.4816 to 0.5465). Versus the dense baseline, 19 of 20 group means are positive. Versus development-tuned α=0.15, 17 of 20 remain positive; collection-level intervals exclude zero for HyDE and Query2Doc. Index-side latency is 0.19 ms/q for one AnchorQE request versus 4.33 ms/q for nine-way CombSUM, with 0.9998 top-10 overlap. Across a 3×3 generator-retriever grid, 44 of 45 cells beat the dense baseline.
QE papers usually compare what to generate. This one isolates how generated text enters a frozen dual encoder. Teams already running HyDE or Query2Doc, and unwilling to rebuild the index, can swap concatenation for vector interpolation. Online calibration needs no labels: eight unlabeled queries set α. Generation quality remains the ceiling. A weak expansion cannot be rescued by a better mixer.
The authors list the caveats. A stream-level α assumes the prefix matches future traffic; a shift makes it stale, and one shared factor cannot treat unusually good or bad expansions. Weak expansions omit information that integration cannot invent; the Qwen3-1.7B band covers the dense baseline less often. The protocol is a no-lookahead split, not live temporal traffic. Ranking equivalence needs a shared dense space and raw dots; mixed sparse-dense streams, truncated ANN lists, and nonlinear fusion sit outside the proof. Prefix calibration costs two probe retrievals before serving returns to one request.