From Understanding to Action: Feedback-Grounded Policy Discovery for Generative Recommendation
Zhi Chen, Minmao Wang, Xingchen Liu, Haoqiang Liang, Huihuang Lin, Likang Wu, Hongke Zhao, Yulong Wang, Shijie Yi, Fei Pan, Peng Jiang
cs.IR
2026-07-30
LLM reasoning sounds right but lacks outcome feedback. Kuaishou splits intent from policy, finds policy via semantic-similarity reward, distills to a light generator: +4.506% online revenue.
In generative recommendation, an LLM can read a user's messy interaction history and reason about what they want now, which sounds useful. But LLMs are never trained with recommendation-outcome feedback, so reasoning that is linguistically plausible does not necessarily lead to a good recommendation. The authors call this mismatch the Understanding-Action Gap.
They split recommendation knowledge into two kinds. Intent knowledge captures what the user currently seeks, inferring from racket, balls, and shoes that they are assembling a tennis kit. Policy knowledge specifies how to act under that intent: the recommendation direction and the rejection boundary. Within the intent-consistent region of tennis accessories, many candidates are semantically plausible (a cap versus a racket bag), and choosing among them requires outcome feedback that language reasoning alone cannot supply. Most LLM-enhanced recommenders stop at intent modeling, never reaching outcome-grounded policy.
The framework has three stages, aimed at LLM-free online inference.
Stage one, intent induction. An LLM intent agent reads history and item metadata, produces a textual intent, and a frozen semantic encoder turns it into a teacher representation.
Stage two, feedback-grounded policy discovery. A policy agent reads historical prefix-successor transitions and generates K=5 personalized policy hypotheses, each specifying a direction and a rejection boundary. To judge which is good, a fixed executor runs two predictions, intent-only and intent-plus-policy, and the advantage A is the difference of their cosine similarity to the true target in semantic space. The authors use semantic similarity as a dense policy-level reward precisely because rank metrics like Recall@K and NDCG@K are too sparse for iterative policy refinement. A feedback agent examines the group evidence, finds patterns, and policies iterate for at most R=2 rounds, keeping only those with verified positive gain over intent-only.
Stage three, dual-space relational distillation. Online serving cannot run the LLM, so knowledge is transferred to a lightweight SID generator. Two latent tokens (intent, policy) are prepended before SID generation, forming an understand-plan-generate pipeline. Distillation aligns first-order user relations and higher-order neighborhood structure (via KL divergence) rather than matching embeddings directly.
Offline on Amazon, TIGER backbone, Beauty: R@5 from 0.0425 to 0.0491 (+15.5%), R@10 from 0.0617 to 0.0739 (+19.8%), N@10 from 0.0333 to 0.0417 (+25.2%); the LETTER backbone improves similarly. The method beats directly using Qwen3.5-122B across metrics.
Online A/B (Kuaishou, 7 days, about 13.25M users, 1.61M items, about 70M interactions): revenue +4.506%, ADVV (advertiser value) +4.621%, both significant at 95% confidence.
Efficiency is part of the point: the base model takes 0.023s per sample, 0.032s with the two intent and policy tokens, while direct LLM inference takes 4.437s (about 137x slower). Ablations clarify contributions: intent helps most at the first SID level (localizing the target's semantic region), policy at deeper levels (fine-grained item discrimination); dropping higher-order relations, intent, or policy from distillation all cost points.
That plausible LLM reasoning fails to convert into good recommendations is a common pitfall when LLMs meet recommendation, and this paper formalizes it as the understanding-action gap and offers an engineerable fix: mine policy knowledge from trajectories, verify it with dense rewards, then distill the LLM away. For industrial recommendation teams, the train-with-LLM-teacher, serve-LLM-free pattern balances quality and latency, and it carries a real large-scale A/B result. The intent-policy split is also a clean modeling abstraction worth borrowing on its own.
Policy discovery and evolution happen only on training interactions; validation and test targets never enter policy generation, evaluation, or refinement. Users without positive-advantage candidates are excluded from teacher-side distillation, and coverage of that population is not discussed. Iteration is capped at R=2 rounds in practice, with no curve for the cost-benefit of more rounds. The online test reports only revenue and ADVV, with no user-side experience metrics such as diversity or novelty. The semantic-similarity reward depends on encoder quality, and a biased reward propagates straight into policy selection, yet robustness of the reward gets little discussion.