Explainable Recommendations at Scale: LLM Rationales for YouTube Music Artist Discovery
Xiao Liu, Yanwei Song, Srivaths Ranganathan, Yuan Chen, Zheyun Feng, Parker Steenburgh, Jochen Klingenhoefer, Nathan Lasche, Gergo Varady, Tim Steele
cs.AI, cs.IR
2026-09-21
Gemini precomputes unheard-artist pools and rationales offline; serving only fetches the cache. A YouTube Music A/B lifts YDD engagement 22.43% and discovery retention 8.07%.
Music apps have to play favorites on demand and still get people to tap artists they have never heard. The second job usually fails on confidence, not coverage. Collaborative filtering emits a score. Heuristic copy says "For Fans of X" or "Sounds like Y", and anything below a cosine threshold never reaches the discovery shelf.
An LLM can say why an unfamiliar artist belongs next to a listener's taste, and it can pull semantic neighbors from a thin history. Generating that live for tens of millions of daily users does not fit a sub-second budget. This Google case study on YouTube Music moves the LLM entirely offline and serves a cached discovery profile.
The system is two-phase. Offline generation fires when an eligible user opens the YouTube Music homepage, not as a daily batch over the whole population. If the profile is not ready, serving falls back to the old heuristic shelf so the first paint does not stall. Once cached, later sessions read it. Idle users never pay for inference.
Gemini reads the full listening history, splits it into taste clusters, nominates likely-unheard artists per cluster, and writes a natural-language rationale in the same pass, which functions as chain-of-thought. A production example under a Stephen Sanchez card: "Reinvents classic mid-century crooner vibes for the modern era like Laufey".
Two guards sit after generation. An LLM-as-a-Judge (Gemini 3.1 Pro) scores cluster cohesion and candidate fit on a 1–5 scale; scores below 3 trigger alternatives and a written failure reason. Recurring failures were style drift, entity hallucination (directors or lyricists nominated as artists), and language mix-ups. Those patterns went into the prompt rather than into a fine-tune, and the pass rate moved from 64.8% to 74.4%. A knowledge-graph canonicalizer then requires every artist string to map to a real entity ID, which drops plausible fakes such as "Gavin DeBardeleben" and Khmer-syllable inventions. A familiarity filter against the full history keeps the shelf actually new; policy-violating copy is dropped.
Online serving does two cheap things: it constrains existing retrievers to songs by profile artists, and it attaches LLM rationales to every eligible candidate, including baseline hits that happen to match. The production YDD stack is the classic two-stage YouTube design: multi-source retrieval, heuristic rationales gated by co-watch or audio cosine, a deep ranker, then packing. Candidates without a rationale do not enter YDD.
Profile jobs run on opportunistic, preemptible TPUs with refresh cycles offset from peak traffic. The paper describes tens of millions of daily active users.
A two-week A/B test compared production heuristic rationales with the LLM pool plus natural-language copy.
| Metric | Change | 95% CI |
| Views from the new nominator | +4.1% | not reported |
| Uniqueness of the new nominator | 74.00% | not reported |
| YDD shelf engagement | +22.43% | [16.93%, 27.93%] |
| YDD discovery retention | +8.07% | [0.90%, 15.24%] |
| Mean online latency | +0.3% | not reported |
A multi-arm holdback splits nomination from annotation. Turning off only the LLM rationales, with UI and candidates fixed, drops engagement nearly as much as removing the whole system. Turning off only the new nominator is engagement-neutral. Discovery degrades under a full holdback and partially when either piece is removed. Rationales drive shelf engagement; discovery needs both. After launch, YDD rose into the home page's top three slots more often.
This is a production recipe, not a new generative retriever. Keep the LLM off the request path and the latency SLA survives. A judge plus a knowledge graph turns hallucinations into a filterable entity problem, and prompt iteration replaces fine-tuning. The holdback is the useful result: copy that explains a stranger is worth more than an extra retrieval source, and heuristic similarity thresholds had been hiding candidates that become clickable once the reason is readable.
For other discovery surfaces, the portable pieces are the offline/online split, entity grounding, and a holdback that separates explanation from nomination.
No absolute engagement rates or sample sizes, only relative lifts. The window is two weeks; novelty decay is unmeasured. Generator and judge are both Gemini, so self-preference is possible, and there is no human-label audit. Only the YDD shelf is tested; harm on the main player, search, or radio is unreported. The +0.3% latency figure is a mean; tail latency is missing. Lazy generation leaves cold users on heuristics for early sessions, with no slice for that cohort. There is no qualitative read on whether the rationales feel accurate or patronizing.