HiGenTrans transfers LLM knowledge into generative recommenders at input, representation and decision levels
_reachsumit · x · 2026-11-04
The HiGenTrans paper proposes a three-level framework addressing two obstacles in industrial generative recommendation: interaction-only supervision underuses pretrained LLM semantic knowledge, and beam-search likelihood ranks token sequences rather than estimating item relevance.
Three levels:
- Input: Global Semantic Priming adds an LLM-derived user representation to the beginning-of-sequence embedding as a semantic prior
- Representation: Hierarchical Representation Transfer aligns intermediate LLM representations with depth-matched GenRec layers via mean-centered projections and gated residual connections
- Decision: Multi-Signal Decision Calibration uses a lightweight mixture-of-experts calibrator to combine semantic relevance, generation confidence, and candidate-set context when reranking beam-search outputs
All semantic features are computed only from observable interaction prefixes, preventing target leakage.
More from Research
- François Chollet: modeling brain wiring as an engineering problem constrained by genome size and time — CatAstro_Piyush · 2026-11-04
- OffsecCyber dataset combines HackerOne reports and CVEs to train pentesting LLMs — AccBalanced · 2026-11-04
- Monofact rate in training data governs LLM scientific discovery, preprint finds — CatAstro_Piyush · 2026-11-04
- qwenTPU compiles a TPU into Qwen3.5 weights, achieving exact integer matmul — CatAstro_Piyush · 2026-11-04
- Jev model improves ICL demonstration retrieval via distillation into a lightweight MLP — kalyan_kpl · 2026-11-04
- Indie researcher open-sources 10 years of math work: 146k files, AI-accelerated over 3 years — HamOnWhy · 2026-11-04