Guiding the coarse levels of semantic IDs makes the fine levels learnable
Bin Wang, Zhengyu Zhang
cs.IR, cs.CL, cs.LG
2026-09-03
Meta's Guided SID forces the coarsest Semantic ID level to encode item attributes like targeting country, lifting recall@10 by 1.39x and MRR by 37% in a matched A/B.
Generative retrieval represents each item as a Semantic ID, a short sequence of discrete codes produced by residually quantizing a content embedding with an RQ-VAE, and trains a language model to autoregressively generate a target item's SID instead of doing dot-product search over millions of items. The paradigm has a structural weakness: SID generation goes coarse-to-fine, so a wrong first code corrupts every subsequent code, yet in standard RQ-VAE that coarse level is just an arbitrary geometric split of embedding space with no semantic identity. The model has to guess a region it can't even name from the user's history alone. The deeper issue is a double misalignment: the tokenizer is trained independently to reconstruct embeddings, so its codes are aligned with neither the downstream LLM vocabulary, since they enter as opaque tokens, nor the actual task. Prior fixes patch this from the consumption side, using alignment corpora, RL, or dedicated encoders to make codes legible to the LLM, or from the tokenizer side, backpropagating the task objective into the tokenizer or supervising each level with a learned tag hierarchy. Either way the recovered meaning is still content-derived and may not be what the task actually needs.
Guided SID forces the coarsest level or two, G={0} in the paper's main experiment, to encode a predefined, text-groundable, task-relevant categorical attribute, targeting country in their ads setting, instead of running nearest-neighbor search. The quantizer simply substitutes the attribute label for the selected index at that level; the codebook itself stays learnable and keeps receiving reconstruction gradients through the straight-through estimator. That guided level becomes 100% accurate by construction, human-readable, and text-grounded: the model reads "United States" in the prompt and just has to look up the matching code, turning what was prediction into lookup.
The piece that makes this workable in practice is trie-merge. Any high-cardinality or set-valued attribute is mapped onto exactly K codes by canonicalizing its values into a trie where sibling nodes share a prefix, country codes sorted by popularity, category taxonomies as paths directly, then repeatedly merging the two smallest sibling nodes bottom-up until only K leaves remain. Because merges only ever combine the most similar neighboring values, this avoids the naive top-K-plus-one-catch-all approach, which would dump the entire long tail into a single meaningless code.
On a 4.4-billion-ad industrial corpus, guiding level 0 on targeting country, with L=6 levels and K=256 codes each:
| Metric | Guided | Vanilla | Prepended | Ratio |
| recall@1 | 0.0166 | 0.0122 | 0.0076 | 1.36x |
| recall@10 | 0.0890 | 0.0639 | 0.0510 | 1.39x |
| MRR | 0.0355 | 0.0260 | 0.0184 | 1.37x |
| attribute match@1 | 0.1770 | 0.0422 | 0.1618 | 4.19x |
Forcing the coarse level onto a non-reconstruction signal doesn't hurt tokenizer quality; reconstruction distance actually comes in 27% lower than the vanilla baseline. A third arm, Prepended, exposes the same attribute as an extra leading token while leaving the content codes untouched. It recovers almost none of the gain, landing below the vanilla baseline on some metrics. That comparison locates the improvement in restructuring the coarse code itself, not merely in giving the model extra context. The unsupervised residual levels, c1 through c5, also improve, by 1.4 to 1.5x across all 24 position-k cells measured, though the effect is far smaller than the 4.19x gain at the guided level itself.
This gives generative retrieval systems a close-to-free structural fix. Any categorical attribute that's content-relevant and can be written into the prompt, targeting country, language, category, can be baked directly into the most error-prone, most consequential code position instead of being left to emerge from an arbitrary geometric split. The trie-merge construction is what turns this from a toy idea into something deployable, since real-world attributes are almost never neatly bounded to a small code budget. For teams already running RQ-VAE plus autoregressive generation for recommendation or ad retrieval, this is a change that grafts onto an existing tokenizer rather than requiring new alignment data for the consuming LLM.
The end-to-end experiment tests exactly one attribute, targeting country, whose codebook is heavily head-weighted: the largest bucket alone holds 22.1% of ads, and the top ten hold 63.6% between them, so a fifth of the corpus sits behind a single code carrying comparatively little information. Country also correlates with content only indirectly, through advertisers tailoring creative to markets, rather than describing content directly. The authors acknowledge a flatter, more directly content-bearing attribute might do better but don't test one. Only a single guiding depth, G={0}, is evaluated; a multi-level guided prefix, such as country then category then subcategory, is untested. Training stops at supervised fine-tuning, so the paper doesn't evaluate guided assignment under RL or in a deployed serving model.