QReason: Query-Focused Decoupled Chain-of-Thought for Efficient Passage Reranking
Yang Zhang, Wenhan Liu, Qiannan Zhu, Mingming Li, Yuanfei Huang
cs.IR
2026-09-25
QReason writes a ranking CoT once and reuses it across sliding windows, lifting Qwen3.5-35B-A3B to 36.93 NDCG@10 on BRIGHT at 7.9x the speed of ReasonRank-32B.
Listwise LLM rerankers that think in public handle hard queries, but the context window cannot hold every candidate, so they slide. Each window regenerates reasoning about the same query. On BRIGHT, ReasonRank's chains for one query across windows exceed 90% semantic similarity under BGE-m3. The expensive part is query-level reasoning; only the local span is actually reading the current passages.
Retrieval-stage rewrites also fail to transfer. Rerankers are picky about fine query-document interactions, and extra tokens become false relevance. GPT-4 and TongSearch rewrites that help retrieval wobble under reranking; GPT-4 drops 2.61 average points on Qwen3-30B-A3B.
QReason peels global query reasoning off the window loop. A rewriter sees only the original query and emits a ranking-oriented reasoning query. A non-reasoning reranker then reuses that string in every window, appended to the original query.
Training has two stages. Relevance-Grounded Supervised Tuning uses DeepSeek-R1 as teacher to synthesize reasoning queries from (query, gold relevant passages), then SFT on Qwen2.5-7B-Instruct. Gold passages are visible only in training, to pin the rewrite to actual evidence. Rank-Aligned Dual-Reward Refinement hides those passages and samples a group of rewrites with GRPO. The listwise reward is NDCG@10 from a frozen reranker. The pointwise reward is the fraction of gold passages judged relevant to the rewrite, so the policy cannot just please one reranker. The two rewards add; the best λ on the sweep is 0.6.
On BRIGHT, QReason plus Qwen3.5-35B-A3B reaches 36.93 mean NDCG@10, above ReasonRank-32B at 36.45 and the original-query setting at 35.49 (+1.44). The same rewrites lift Qwen2.5-32B-Instruct from 32.75 to 35.18 (+2.43) and Qwen3-30B-A3B from 34.48 to 35.53 (+1.05). GPT-4, DeepRetrieval, and TongSearch rise on some rerankers and fall on others.
Ablations on Qwen2.5-32B-Instruct: full 35.18; RGST only 33.50; RADR only 33.91; SFT without gold passages 32.98; drop pointwise or listwise and the score lands at 33.85 and 33.90. Both stages and both rewards earn their keep.
On six BRIGHT subsets, end-to-end latency versus ReasonRank-32B, rewrite included, is about 7.86x with the MoE reranker Qwen3.5-35B-A3B and 2.58x with Qwen2.5-32B-Instruct. The saving is skipping autoregressive CoT on K−1 windows, not the extra prefill tokens.
If CoT reranking is on the table for reasoning-heavy search and sliding-window latency is not, this split is concrete: think about the query once, rank inside the window. A 7B rewriter can feed 30B-class non-reasoning rerankers and slightly beat a 32B reasoning reranker on the mean.
The gain is a point or two of NDCG, not a generation change. Rewrites can still add unsupported content; the case study shows QReason dragging in national accounts and expenditure-survey language.
Rewards cover listwise NDCG and pointwise relevance, not coverage, faithfulness, or redundancy; the authors flag rubric rewards as future work. The main arena is BRIGHT reranking of ReasonIR top-100; FreshStack and NeuCLIRBench sit in the appendix. The rewriter is distilled from DeepSeek-R1 with gold passages in the SFT target, so "query-only" inference still has a train-test gap. Rewrites were not plugged into specialized rerankers such as RankT5 or ReasonRank, which were trained on original-query inputs.