From Interests to Semantic IDs: Retrieval-Grounded Credit Assignment for Generative Recommendation
Mengdan Zhu, Yufan Zhao, Yao Zhao, Sophie Di, Tao Di, Yulan Yan, Sridhar Iyer, Liang Zhao
cs.IR, cs.AI
2026-09-24
A frozen retriever scores each generated interest; only queries that hit the target get positive advantage. Video Games Recall@10 rises from 0.0856 to 0.1195 vs SID-plus-trace RL.
Generative recommenders encode each catalog item as a short Semantic ID (SID) so a language model can emit the next item token by token. Reasoning-enhanced variants first write a textual trace, then beam-search a SID. Training typically uses group-relative policy optimization with an exact-match SID reward. In a large catalog that reward is sparse.
Two failure modes follow. If every rollout in a group misses the target, group-normalized advantage is zero and nothing learns. If several traces share the same SID reward, they get the same advantage however different the text. The reward sees the decoded SID and never the interest hypotheses that produced it. Credit assignment stops at the item tokens.
Each trace is forced into a fixed schema: a history summary, several future-interest lines, then a valid SID. A frozen catalog retriever runs every interest as a query. The rollout-level retrieval reward is 1 if any query retrieves the target in the top-K. Per-query hit indicators are kept for routing.
Three binary rewards are standardized separately inside each group: trace validity, retrieval coverage, and SID exact match. Retrieval advantage is applied only to interest lines. Hitting lines share the positive signal; on a miss, the negative signal is shared across all parsed interest lines. The final SID span never receives retrieval credit. SID sampling is constrained by a catalog prefix trie.
Training has three stages on Qwen3-1.7B with a frozen three-level RQ-VAE tokenizer (256 entries per codebook). Stage 1 aligns SID tokens with text. Stage 2 uses GPT-4o traces to teach the schema. Stage 3 runs the RL above. Default K is 50, the retriever is frozen Qwen3-Embedding-4B, and each group has 16 rollouts. At inference the same interests can form an auxiliary recall pool and optionally shrink the SID trie.
Amazon Reviews, catalog-constrained beam search, no sampled negatives:
| Setting | Video Games R@10 / N@10 | Office R@10 / N@10 | Industrial R@10 / N@10 |
| SIDReasoner | 0.1031 / 0.0563 | 0.1648 / 0.1208 | 0.1438 / 0.1010 |
| Stage-3 SID+trace control | 0.0856 / 0.0493 | 0.1512 / 0.1124 | 0.1387 / 0.0953 |
| Hit-query retrieval reward | 0.1195 / 0.0796 | 0.1765 / 0.1332 | 0.1602 / 0.1144 |
Versus the matched SID-plus-trace control, Recall@10 rises 15.5%-39.6% relative and NDCG@10 rises 18.5%-61.5% across the three domains. SID-only or SID-plus-trace RL stays near the Stage-2 SFT checkpoint. All 12 metrics are best in the table.
During training, 69.1%-80.8% of groups have constant SID rewards and therefore zero SID advantage. Retrieval reactivates a large slice of those groups: 37.8%, 37.7%, and 19.6% in the last window on Games, Office, and Industrial. Broadcasting retrieval advantage over the full response or the whole interest block loses to hit-query routing. Against block routing, hit-query routing adds 11.4% Recall@10 and 30.5% NDCG@10 on Games.
Overlap is partial: 25.91% of targets missing from the SID top-10 are still retrieved by some interest at 50. An oracle that picks the target-relevant generated interest and constrains SID decoding reaches 14.93 Recall@10, above 11.95 for global SID decoding. A random interest is substantially worse.
If a reasoning recommender is still trained on "did the final SID match", the interest lines are mostly unused. Making those lines executable queries turns a frozen retriever into process supervision, without a separate process reward model. Teams already on SID plus GRPO can change the credit routing and leave the decoder interface in place. The recall pool is a side product. The oracle gap shows that choosing which interest to trust is still open.
Catalogs have a few thousand SKUs and histories are truncated to 10 interactions, far from industrial scale. The reward retriever and the interest-recall metric share Qwen3-Embedding, so Interest Recall gains are not an independent generalization test. Stage-2 traces are written by GPT-4o that sees next-item metadata; teacher leakage is not isolated. The candidate-constrained SID upper bound uses an oracle interest picker; the paper flags selection without the target as future work. Generated traces are optimized for retrieval utility and should not be shown to users as faithful explanations.