Distilling discarded RQ residual trajectories into generative retrieval sweeps all 15 ESCI metrics

Residual Trajectory Distillation for Generative Retrieval

Weihao Shen, Wei Chen, Fuwei Zhang, Guojun Liu, Qingsong Hua, Wei Lin, Fuzhen Zhuang

cs.IR

2026-09-30

ResTD distills residual trajectories discarded by RQ indexing into generative retrieval training, first on all 15 ESCI metrics across three languages at zero inference cost.

What problem this solves

Generative retrieval treats search as sequence generation: given a query, the model autoregressively writes out a discrete Semantic ID (SID) for the target product or document. Most systems build these IDs with residual quantization (RQ), which picks the nearest codeword at each level and passes the remaining residual to the next. Training then supervises only the final code sequence. That argmin step flattens real information: two items can land on the same codeword through very different residuals, with different orderings and margins over competing codewords, and each residual also determines the input to the next quantization level. The indexer computes all of this and then throws it away. The paper calls this the gap between indexing and retrieval training.

Method

ResTD turns the frozen RQ indexer into a process teacher. The deployed index never changes.

• Teacher: before level t, the residual is scored against every codeword in the codebook; a softmax over negative squared distances yields a full distribution that keeps the ordering and margins among competitors, of which the selected code is just the mode.
• Student: the decoder state that predicts code t is mapped by a learned projection into the residual space and scored against the same frozen codebook. Teacher and student share the codebook geometry and are aligned with a KL loss.
• Multi-horizon supervision: the same state is also aligned with teacher distributions from later levels (t+1, t+2), before those codes enter the autoregressive context. The supervision is triangular, so early states answer for a longer suffix; distant horizons are discounted by ρ, 0.7 by default.
• Collision-aware correction: collision resolution can rewrite stored SIDs, so residual trajectories are recomputed along the stored path, and a minimal interpolation enforces that the stored code stays the teacher's top choice without disturbing ratios among the alternatives.

The auxiliary loss enters through a linear warmup and is capped at 5% of the SID loss. At inference the teachers and heads are removed; the index, decoding distribution, and trie-constrained beam search (beam 100) are untouched.

Results

On ESCI product search in English, Spanish, and Japanese (T5-base / mT5-base; 4 RQ levels, 256 codewords each):

It ranks first on all 15 metrics (3 languages by 5 metrics) against BM25, BGE-M3, TIGER, MERGE, HGRec, and others. Gains concentrate at early ranks; the relative gain on Recall@100 shrinks to 3.6-6.4%.

The matched controls are more informative than the leaderboard. From the same reproduced CaLIR checkpoint, a 1,200-update budget, and three paired seeds: plain continued training reaches R@10 12.82, current-level distillation only 13.23, full multi-horizon 14.34. Swapping future soft targets for future hard labels drops it to 13.56; codebook-only soft targets give 13.48; a shuffled teacher gives 12.86, indistinguishable from doing nothing. So the gain tracks the residual geometry itself, not soft targets in general. Probes on frozen retrievers back the mechanism: future codebook preferences become 5.3-13.8% more recoverable from early decoder states, while current-level targets move only 0.1-0.3%.

Transfer holds up elsewhere. On NQ320K, Recall@10 on unseen queries is 4.79 against CaLIR's 0.51. Plugged into TIGER-style recommendation, Yelp Recall@10 reaches 4.87 (ETEGRec 4.15) and Beauty 7.28 (best baseline 6.15). Training cost is 1.12x per update, +31.8 MB peak memory, and 0.09 GB of precomputed residuals; inference overhead is zero.

Why it matters

For teams already running RQ-based generative retrieval or recommendation, this is a low-risk, training-side change: keep the deployed index and decoder, add a few projection heads and one auxiliary loss. The supervision signal was already computed during indexing and then discarded. It is an incremental improvement rather than a new paradigm, but the controls are unusually thorough, and the finding that residual-induced preferences beat both hard future labels and codebook-only soft targets is a useful data point for anyone designing SID training.

Limitations

• Most Table 1 baselines use published numbers under their original protocols, so they are not strictly comparable; the matched controls are the real evidence, and there the absolute R@10 gain is about 1.5 points.
• The method requires access to the RQ indexer's residuals and codebooks. A third-party index with a black-box quantization process cannot be supervised this way.
• The authors themselves report that the relationship between teacher disagreement and gains is dataset- and metric-dependent with no uniform pattern; on ESCI-ES, NDCG gains peak in the middle-disagreement group.
• The hyperparameter surface is nontrivial: horizon H, teacher and student temperatures, discount ρ, collision-correction floor, and loss weight all matter; the appendix covers sensitivity, but a new dataset means retuning.
• Recommendation gains are uneven: about 1.7% relative on Instruments, with the large gains on Yelp and Beauty.

Terms

• generative retrieval: retrieval reformulated as autoregressive generation of discrete IDs instead of nearest-neighbor search over embeddings.
• Semantic ID (SID): a sequence of discrete codes quantizing an item representation; both the generation target and the retrieval address.
• residual quantization (RQ): vector quantization applied level by level to the remaining residual after each codeword choice.
• knowledge distillation: supervising a student with a teacher's soft distributions rather than one-hot labels.
• trie-constrained beam search: beam search restricted to valid SID prefixes stored in a trie.

Source

• arXiv 2609.39319v1
• PDF
• Hugging Face

What people are saying

• ResTD: Distilling Discarded Residual Trajectories into Generative Retrieval Training _reachsumit

Related papers

• Re-judging TREC DL 2019 keeps rank τ at 0.88 while nDCG@10 nears the human ceiling of 0.81
• Google and University of Toronto Propose a Three-Axis Taxonomy to Make GEO Benchmarks Comparable
• FLINT pools both sides to 32 vectors, 1.5–2.8× faster than ConstBERT on long queries
• 76% of items saturate after one layer; LoopRec routes depth per item
• GPT-6 Astra hits 39% on 200 live-web deep-search tasks; humans score 21.1%
• QD-Rec leads mean LLM-judge quality in most settings and usually builds the biggest archive

All paper explainers