ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling
Vaibhav Singh, Soumya Suvra Ghosal, Sarvesh Gharat, Soumyabrata Pal, Ramasuri Narayanam, Dinesh Manocha
cs.AI
2026-08-11
ThinkRetrieve retrieves a solved example at each reasoning step and injects it into the trace, beating sequential test-time scaling on every model-benchmark cell, up to 60% relative on AIME 2025.
Large reasoning models trade more thinking for higher scores: give the model a bigger reasoning budget and let it spin out a long chain of thought. But a wave of recent work shows this sequential test-time scaling has diminishing or even negative returns. Longer traces mean more uncertainty, error compounding, and drift off the original problem. On a hard benchmark like AIME 2025, sequential TTS plateaus around 22% and stops improving even as the budget grows.
ThinkRetrieve retrieves, at each reasoning step, a similar already-solved problem and injects it into the current chain of thought. The difference from ordinary RAG is that it retrieves examples of how to reason, not relevant facts.
Each step runs as follows: the model reasons normally and emits an intermediate answer at a step boundary; the original problem and that intermediate answer are combined into a query; an E5-Large encoder runs nearest-neighbor search over a corpus of solved examples; the retrieved example is formatted as [Example: problem Solution: solution] and inserted into the trace; the model continues reasoning with that example in context.
The corpus is about 310,000 examples filtered from the synthetic portion of NuminaMath-1.5, decontaminated in two stages (exact-match plus cosine-similarity above 0.90), with confirmed zero overlap against the four test sets.
Across five reasoning models (DeepSeek-R1-Distill-Qwen-1.5B up to Qwen3-8B) and four benchmarks (GSM-8K, MATH-500, AIME 2025, SciQ), ThinkRetrieve beat sequential TTS on every model-by-benchmark cell. The largest gains came on the hardest benchmark, AIME 2025:
| Model | Sequential TTS -> ThinkRetrieve |
| Qwen3-1.7B | 22.2% -> 35.6% (+13.4 absolute, 60% relative) |
| Qwen3-4B | 64.4% -> 66.7% |
| Qwen3-8B | 68.9% -> 71.1% |
On GSM-8K, DeepSeek-R1-1.5B's sequential TTS collapsed from 83% to 52% as the budget reached 22K tokens, while ThinkRetrieve held at 84%. In a compute-matched comparison on a MATH-500 subset, a single ThinkRetrieve pass (0.47) beat self-consistency with k=8 samples (0.20). The cost is one retrieval call per step, roughly 6% extra wall-clock time.
Test-time scaling is a crowded frontier, and "think longer" has run into a wall. This paper offers a different route: instead of letting the model spin ever longer, hand it a ready example when it gets stuck. Retrieving per step is more accurate than static RAG (which only seeds examples up front), because retrieval tracks the reasoning as it evolves. The method is not bound to math; any reasoning task where you can assemble a high-quality example corpus could adopt it.
The effect depends heavily on corpus coverage: on out-of-distribution problems the retrieved example may be irrelevant or misleading, and ThinkRetrieve's entropy-reducing property can then lock the model onto a confident wrong answer. The authors concede about 6% latency per step and that they cannot fully rule out latent structural leakage, two problems with different surfaces but the same solution procedure. Whether the gains hold on tasks where building a high-quality corpus is hard, such as code generation or open-ended logical reasoning, remains an open question.