Think Thrice Before Reranking: Multi-perspective Evidence and Reasoning Integration for Text Reranking
Lijun Liu, Zhengzong Chen, Wenyan Li, Yuanyuan Zhao, Fei Huang
cs.IR, cs.CL
2026-09-17
Honor's MERIT-Rank runs semantic, intent, and evidence paths then fuses them. The 4B model hits 36.6 NDCG@10 on BRIGHT, ahead of ReasonRank-7B (35.7) and ERANK-4B (30.5).
LLM listwise rerankers such as Rank-R1, REARANK, and ReasonRank almost all write one reasoning chain, then emit a ranking conditioned on that chain. A local error in the chain travels through autoregressive decoding into the final order. Relevance is also multi-sided: topical overlap, whether the document actually answers the user's intent, and whether the text supplies checkable evidence often pull in different directions. One trajectory usually captures only a slice.
Honor Device frames the job as two coupled problems: how to run complementary trajectories inside a single model, and how to train that longer generation without it collapsing.
MERIT-Rank builds a Multi-Trajectory Reasoning Space (MTRS) with three fixed views:
The model writes a reasoning-plus-ranking block per view, then a synthesis block that folds the three into a final order. One model, one autoregressive pass. No separate ensemble.
DeepSeek-R1 is the teacher. Starting from 31k queries drawn from MS MARCO and prior reranker corpora, it writes the multi-view traces and the fused gold ranking; GPT patches format failures. Dual filters keep a sample only if the fused ranking beats every single-view ranking on NDCG@10 and places labeled relevant documents at the top. That leaves 21,032 examples: 10k for SFT, 11k for RL.
Long chains invite a shortcut: skip the views and dump a ranking. Progressive Rank Policy Optimization (PRPO) is the curriculum that fights that. SFT first locks the format. Two GRPO stages follow. Stage one rewards the tag structure (<multithink>, <synthesisthink>, <answer>), NDCG/MRR gains over the initial retrieval, and Rank-Biased Overlap with the gold order. Stage two adds absolute MRR and NDCG. Relative gains stabilize first; absolute metrics push later.
On BRIGHT, all systems rerank ReasonIR's top-100. Metric is NDCG@10:
| Method | Avg |
| ERANK-4B | 30.5 |
| Rank-K-32B | 32.6 |
| ReasonRank-7B | 35.7 |
| MERIT-Rank-4B | 36.6 |
| MERIT-Rank-7B | 37.1 |
| MERIT-Rank-32B | 40.3 |
The 4B model is 6.1 points above same-size ERANK and slightly above ReasonRank-7B. On classic TREC, MERIT-Rank-7B scores 74.1 / 75.3 on DL19 / DL20 against ReasonRank-7B's 69.8 / 71.8; the seven-dataset BEIR mean is 60.0 vs 55.4.
Ablations on the 7B BRIGHT run: drop synthesis and the score falls to 35.29 (-1.84); drop all reasoning and it falls to 32.18 (-4.95); drop P-GRPO and it falls to 34.78. Four unconstrained "general" traces score 29.0; the explicit three-view plus synthesis setup scores 34.1. Under sliding windows, three windows reach 35.81, matching ReasonRank's 35.74 with nine windows, so total tokens go down even though each step is longer.
If you want reasoning-based rerank without defaulting to 32B, this is a "write the views into the format, then curriculum-train it" recipe. A 4B model beating most larger rerankers on BRIGHT is evidence that the gain sits in structured multi-view reasoning more than in parameter count. Keeping fusion inside one model also avoids an ensemble at serve time. Long candidate lists, where window count explodes, are the practical win.
It is still an incremental step: the teacher is DeepSeek-R1, the data is MS MARCO-family, and the eval protocol matches ReasonRank and REARANK.
There is no dedicated limitations section. Visible gaps: synthetic traces depend on a strong teacher, so reproduction is expensive; per-step generation is longer, with latency recovered only by using fewer windows; the 4B model scores 33.0 on the theorem subset TheoT, below Rank-K-32B (41.0) and ReasonRank-7B (41.8), so extra views do not obviously help formal content. BRIGHT first-stage retrieval uses GPT-4 expanded queries, matching prior papers, which inflates absolute reranker scores. Training queries are said to exclude eval sets; leakage risk still depends on how clean those "existing reranker datasets" actually were.