Not Worth Another Token: Marginal Value Estimation for Efficient Deep Research Agents
Harshitha Kolukuluru, Reshma Ashok, Kirat Arora, Evan William Ciccarelli, Nischal Ashok Kumar, Lunyiu Nie, Franck Dernoncourt, Samyadeep Basu, Ryan A. Rossi, Nedim Lipka
cs.AI, cs.IR, cs.MA
2026-08-09
A baseline deep-research run burns 375k tokens per report; lightweight pruning cuts that to 100k with little quality loss, and where you prune matters more than which score you use.
Long-horizon research agents iterate retrieval, aggregation, and synthesis around an open question, and context keeps growing while the marginal value of each extra piece of evidence declines. The fallout is more tokens burned, longer latency, and noisier input fed to the final report. The authors give a baseline number: an unpruned deep-research pipeline averages 29 nodes explored, 375k tokens, and 3,422 seconds per report. The question is direct: can low-value content be removed earlier.
The authors formalize "is this evidence worth keeping" as a single scoring function V(x | Ct, Q): given query Q and accumulated context Ct, does candidate x still add enough value. Pruning can be inserted at three pipeline stages:
V can be instantiated many ways. Lightweight heuristics make no LLM call and add no inference cost: MMR (balancing relevance against redundancy), GRN (residual norm for new semantic directions), CD (centroid drift), DPP (determinant-based diversity), SC (token-normalized coverage), plus hybrid combinations and lexical variants that swap dense embeddings for TF-IDF. The authors also try two heavier options: an LLM acting as the pruning judge, and a multitask neural value model trained on execution traces.
The benchmark is DeepResearchGym, 100 queries from Researchy Questions run on the GPT-Researcher framework. Three core findings:
| Stage config | Best-efficiency point | Tokens | Quality (overall) |
| One-stage, post-retrieval | MMR | 114.6k (-69.5%) | 56.62 (baseline 57.83) |
| Three-stage | MMR | 100.1k (-73.3%) | 55.90 |
| One-stage, pre-synthesis | Hybrid | 332.3k | 60.68 (+2.85, best quality) |
The most token-efficient three-stage MMR squeezes 375k down to just over 100k while quality only drops from 57.83 to 55.90. The most counterintuitive and most important result: where you prune matters more than which scoring rule you use. Early pruning (pre- and post-retrieval) saves real retrieval and processing cost, while pre-synthesis pruning mostly refines the final context. Lightweight heuristics cut up to 73% of tokens with almost no quality loss, the learned neural model only ties heuristics on a few trade-off points, and no single method leads on quality, efficiency, and faithfulness at once.
The authors flag one trade-off explicitly: pruning can preserve report quality while discarding the evidence needed for maximal relevance. No single-stage pruning method beats the baseline on the KPR+KPC relevance metric.
For anyone building long-horizon agents, this paper hands over engineering conclusions you can apply immediately. Do not start by agonizing over a fancy scoring function. Insert pruning at the two early stages (pre- and post-retrieval), run a zero-cost heuristic like MMR, and you will most likely cut the bulk of tokens without losing quality. It also makes one thing clear: under pruning, a fluent report and a complete evidence base diverge. Maximum compression means accepting some quality loss; maximum quality means giving up maximum compression.
The authors list several. The findings come from a fixed GPT-Researcher pipeline and DeepResearchGym, so trade-offs may shift under different agent architectures, retrieval systems, or underlying LLMs, and method-level quality rankings are less stable across benchmarks than the efficiency findings. Quality scores come from an LLM judge, and absolute scores shift substantially when the judge changes, so quality-sensitive conclusions are best read as relative comparisons under a fixed judge. Every method needs a threshold; the authors report local sweeps, but finer method ordering can still shift. Automatic metrics only partially reflect factual correctness and real usefulness to users. There is an ethics note too: pruning can systematically discard contradictory evidence, minority viewpoints, and important caveats, leaving a fluent but under-supported summary, so it should be paired with source transparency and human oversight.