Seek: Self-Evaluative Exploration for Knowledge Retrieval
Amin Bigdeli, Radin Hamidi Rad, Negar Arabzadeh, Sajad Ebrahimi, Hai Son Le, Charles L. A. Clarke, Ebrahim Bagheri
cs.IR
2026-09-24
Seek iterates at test time: pseudo-passages, a fresh corpus pass, and graded LLM judgments. Qwen2.5-7B hits 30.9 nDCG@10 on BRIGHT vs BM25 17.0 (+82%), beating trained rerankers.
LLM retrievers and rerankers still mostly touch the corpus once. BM25 pulls a top-100 pool; a reranker reorders that pool. Relevant documents that never entered the pool stay gone. Dense retrievers spend more compute up front and typically still retrieve in one pass. Reasoning-intensive queries, with lexical mismatch and implicit constraints, make that one-shot recall ceiling especially thin.
Agentic deep-search systems rewrite queries, inspect evidence, and search again, but they usually aim to write an answer. Seek outputs a ranked document list. The metrics are nDCG and Recall on ad-hoc retrieval over a fixed corpus.
Seek is training-free and runs a three-stage loop at test time for up to T=5 rounds. One LLM is both generator and assessor. The paper reports Qwen2.5-7B-Instruct, Qwen3-4B, and GPT-4.1.
Early stop fires if every current top-10 document has the maximum grade, or if the top-10 set is unchanged across two rounds. The final list is two-tier: assessed documents first by grade, ties broken by the best retrieval score across rounds; unassessed documents follow, ordered by that same score.
Primary metric is nDCG@10. BRIGHT uses seven StackExchange domains; TREC DL uses MS MARCO (8.8M passages).
| Method | Backbone | DL19 | DL20 | BRIGHT mean |
| BM25 | - | 50.6 | 48.0 | 17.0 |
| ReasonIR-8B | LLaMA-3 | - | - | 24.8 |
| TFRank-7B | Qwen2.5 | 66.5 | 59.3 | 26.8 |
| ERank-4B | Qwen3 | 71.1 | 66.7 | 27.3 |
| RankGPT | GPT-4 | 75.6 | 70.6 | 24.7 |
| Seek | Qwen2.5 | 71.6 | 65.1 | 30.9 |
| Seek | Qwen3 | 66.3 | 67.0 | 33.3 |
| Seek | GPT-4.1 | 73.8 | 71.0 | 37.4 |
Qwen2.5 Seek is an 82% relative lift over BM25's 17.0 and beats every listed trained reranker with the same backbone. Qwen3 Seek is 22% relative above ERank-4B's 27.3. GPT-4.1 is highest on six of seven BRIGHT domains, 37% relative above the strongest baseline. On TREC, Seek matches trained rerankers on nDCG and does not win there. The structural win is Recall@100, which rerankers cannot raise. Biology recall moves from 42 to 82, Psychology from 38 to 72.
On BRIGHT-Biology, BM25 starts at 18% nDCG and 22% recall. Round one reaches about 40% / 43%, round two about 45% / 53%. After round three, nDCG plateaus near 46% while recall crawls to 58% by round five. Swapping in ReasonIR-8B as the first-stage retriever takes Seek from 24.8 to 32.3; Seek with BM25 already sits at 30.9, so the loop dominates the choice of first-stage retriever.
Reasoning retrieval is often bottlenecked by the candidate pool, not by whether the reranker can think. Seek spends test-time rounds on the corpus instead of training a reasoning reranker, and with the same backbone, no finetuning, it passes Rank-R1, TFRank, and ERank. The cost is generating pseudo-passages and judging new documents every round, which is steeper on TREC-scale corpora. If a product already accepts LLM relevance judgments, the same loop can sit in front of a deep-search agent's evidence stage.
Dollar and token costs are barely reported: round caps and 10 judged documents per round, nothing else. The assessor is the generator, so wrong grades contaminate the next query, with no human calibration. An 82% relative lift over BM25=17.0 is loud; 30.9 absolute nDCG is not a solved task. On TREC, nDCG does not beat the strongest rerankers; the home ground is reasoning-heavy lexical mismatch. Figure 3's body text says Qwen2.5 while the caption says Qwen3, so the round curves need a caveat. There is no experiment that plugs Seek into a full QA agent; that retrieval-component story is still a claim.