SelfDR: Self-Distillation from Reasoning for LLM-Based Recommendation
Chumeng Jiang, Jiayin Wang, Xinjie Lin, Zhiqiang Guo, Hengliang Luo, Min Zhang
cs.IR
2026-09-03
SelfDR trains an 8B reasoner with GRPO using recommendation accuracy as reward, then distills the reasoning-augmented teacher into a single-token student that ranks first on all three datasets at 109 ms per query, about 6x faster than RL baselines.
The strongest LLM recommenders now reason before they recommend: multi-step pipelines that extract preferences and build profiles first, or single-step models that emit a long reasoning trace before the answer. Both are more accurate, and both are too slow for production. Generating reasoning tokens per request pushes latency to 500-660 ms in this paper's measurements, which no real-time recommender can absorb.
SelfDR asks whether the accuracy gain from reasoning can be banked without paying the inference cost.
The framework runs on a single LLaMA-3.1-8B in two stages.
First, a reasoner is trained with GRPO. It takes the user history plus the ground-truth next item and explains why the user would click it. The reward is downstream recommendation accuracy, so the generated rationales stay aligned with the task rather than drifting into generic commentary. A masking step removes any span of three or more consecutive words that exactly overlaps the item title, blocking the shortcut of restating metadata.
Second, the rationales are concatenated into the input of a teacher recommender, and a structurally identical student learns from the teacher without seeing any rationale. The distillation target is the teacher's probability distribution over candidate identifier tokens, matched with reverse KL divergence, not its generated text. A dynamic weight α balances this against cross-entropy on ground-truth labels, leaning on the teacher when its ranking of the positive item is strong and backing off when it underperforms the student.
At inference the student emits one character identifier and nothing else.
Reranking SASRec's top-20 candidates on Amazon Clothing, Home, and ML1M:
| Method | Clothing H@1 | Clothing N@5 | ML1M H@1 | Latency (ms) |
| Best traditional | 0.0079 | 0.0200 | 0.0640 | - |
| COT4Rec (multi-step) | 0.0100 | 0.0186 | 0.0575 | 250.0 |
| RecR1 (RL reasoning) | 0.0114 | 0.0191 | 0.0515 | 665.1 |
| SelfDR | 0.0132 | 0.0228 | 0.0845 | 109.5 |
Two findings stand out. On the training side, the GRPO-trained 8B reasoner beats GPT-4o-mini, Claude-3-Haiku, and DeepSeek-V3 as the rationale source, and GPT-5-Chat does too (teacher H@1 0.552 vs 0.429). Rationales from a much larger model can contain knowledge the student cannot absorb; a self-trained reasoner matches its distribution. On the serving side, 109 ms ties SOFT for the fastest method in the comparison, about 6x faster than RecR1.
This decouples the accuracy benefit of reasoning from its inference cost. For recommender practitioners, it is a concrete recipe for banking reasoning gains into a model with zero added latency. The broader pattern, reason during training and not at inference, transfers to any latency-sensitive LLM deployment. One pragmatic ablation detail: continuing supervised-only training overfits, and distillation-only underperforms the dynamic mix of distillation and labels.
Only an 8B backbone is tested; other scales are unverified. The reranking setting uses 20 candidates, and direct full-catalog ranking is not reported. GRPO training still costs 54 GPU-hours in total, comparable to baselines but not cheap. The GPT-5-Chat comparison rests on 2,000 sampled training instances. The dynamic weighting adds four hyperparameters, and although sensitivity analysis looks flat, they would need retuning on a new dataset.