ResTD: Distilling Discarded Residual Trajectories into Generative Retrieval Training
_reachsumit · x · 2026-10-01
A new arXiv paper introduces ResTD, a Residual Trajectory Distillation framework for generative retrieval.
- Generative retrieval builds Semantic IDs (SIDs) via residual quantization, but standard training supervises only the selected codes, discarding the residual trajectories produced during indexing; hard SID supervision collapses distinct quantization behaviors into identical targets.
- ResTD treats the frozen RQ indexer as a process teacher, distilling residual-induced codeword preferences into SID-decoding states so earlier decoder states capture information about subsequent quantization decisions.
- The approach preserves the original retrieval index and inference procedure while enriching training signal, with consistent gains on multilingual e-commerce retrieval.
More from Research
- EIG publishes most rigorous analysis yet of AI's real effect on US entrepreneurship — robseamans · 2026-10-01
- AI "mind-reading" tool reconstructs what you see from brain scans — ChuckDBrooks · 2026-10-01
- Training Qwen3-VL-4B into an AR image generator with Cosmos tokenizer — ostrisai · 2026-10-01
- CMU releases CUA-SWE, a benchmark uniting computer-use agents with visual software engineering — CarnegieMellonU · 2026-10-01
- New paper: principal component regression beats every monotone spectral filter, incl. ridge and GD — kfountou · 2026-10-01
- MIT's xvr matches X-rays to 3D scans in seconds with sub-millimeter precision for surgery — MacrinePhD · 2026-10-01