Paper reuses pretrained SAE features as item vocabulary for explainable generative recommendation
_reachsumit · x · 2026-10-29
The paper "The Vocabulary Is Already Here" proposes the API framework, reusing pretrained sparse autoencoder features (which carry human-interpretable semantic annotations of PLM representations) as the semantic ID vocabulary for generative recommendation (GR), avoiding costly vocabulary fitting.
- Existing GR methods fit SID vocabularies to PLM item representations, adding training cost and limiting coverage to the fitting domain.
- Challenges addressed: preserving item distinctions under compact token budgets, favoring target items over candidates with shared dominant features, and extending interpretability from item content to retrieval decisions.
- PIDs (Pretrained Interpretable IDs) retain selected feature indices plus relative activation strengths for discrimination.
- A Semantic Prediction Adapter (SPA) learns user preferences from interaction histories via joint feature-index and strength prediction, supervised by PIDs.
Code repository linked.
More from Research
- Granary-v2 open speech model expands to 48 languages with open data and pipeline — rdesh26 · 2026-10-29
- Tinker launches Dimuon, a Muon-based optimizer that beats AdamW on LoRA fine-tuning — xennygrimmato_ · 2026-10-29
- Six CUDA kernel optimizations squeeze another 1.78× speedup out of 3DGS training — gkopanas · 2026-10-29
- Cross-lingual LLM effects paper wins Outstanding Paper award at EMNLP 2026 — anas_ant · 2026-10-29
- Lie detector compiled into Qwen3.5 weights flags 91% of lies with zero false positives — teortaxesTex · 2026-10-29
- Practitioner: No serious lab actually trains on the test set today — nrehiew_ · 2026-10-29