QD-Rec: Open-Ended Evolution of Recommendations via Quality-Diversity
Takashi Ishida, Taro Makino, Issa Sugiura
cs.IR, cs.AI, cs.LG, cs.NE
2026-10-16
QD-Rec pairs a diversity-aware bandit with evolved hypotheses. In 1,000 steps it leads LLM-judge quality in most settings, Disney M&A from 2.40 to 2.91; human ratings stay tied.
High-stakes choices often come with a closed menu. An acquirer picks among companies that already exist, a marketer among people already on file, a reader among papers already written. The reason for picking is not closed. The same company can be worth buying for a missing capability, for distribution, or for vertical integration, and those arguments do not run out.
A ranked list assumes each item already has a score. Here the score exists only after a natural-language hypothesis is written and judged. QD-Rec has to return, for one decision-maker profile and a fixed pool, a set of strong item-hypothesis pairs that differ from each other. A hypothesis is a short case for why that item fits that person.
The archive is the answer. Each entry stores an item, a hypothesis, an embedding, and a quality score. Embeddings concatenate item text with the hypothesis, using text-embedding-3-small. Neighbors are cosine nearest neighbors.
Item selection is a diversity-aware contextual bandit. Bayesian linear regression with Thompson sampling draws a quality estimate, then adds cosine distance to the nearest archived item. The diversity weight β is 1. An item already in the archive gets no bonus, so it is rarely chosen again. Across five seeds, no finished QD-Rec archive contains the same item twice. The posterior is warm-started from 10 seed entries.
The proposer sees the profile, the item, and 3 hypotheses drawn uniformly from the archive. It must mutate one prior angle onto the new item, cross several angles together, or explore an angle the archive does not yet contain. The archive is the population.
Admission is local. The judge sees the profile, the new pair, and 3 neighbors, and returns a yes-or-no novelty decision. Novel pairs enter at once. Otherwise the new pair meets only its nearest neighbor in a blind duel: order shuffled, scores hidden, the judge not told which side is new, ties disallowed. The winner stays. Whether or not the pair is kept, the bandit updates on that score. Splitting the two questions stops the judge from treating newness itself as a virtue, and it skips the duel whenever the pair is novel.
Novelty drops both the bandit and the exemplars: random items, zero-shot hypotheses. Novelty+Evo keeps the 3 exemplars and still picks items at random. All three share admission. The main proposer is claude-sonnet-5; a weaker run uses claude-haiku-4-5. The judge is gpt-5.6-terra, a different model family, to avoid self-preference. Each configuration spends 1,000 evaluations and 5 seeds.
Four pools: 1,761 picture books and 1,117 series or young-adult books, scored by the reader's own 1-to-5 stars, which never enter the prompts; 6,332 US companies, with Disney or JPMorgan as the acquirer; 10,000 synthetic California customers, for a fictional desktop 3D printer or an online course; 17,511 accepted papers from ICLR 2026, ICML 2026, and NeurIPS 2025.
All four metrics are computed on the final archive. Quality is the mean score, 0-10 for the LLM judge and 1-5 stars on Goodreads. Diversity is one minus mean pairwise cosine similarity. Coverage is occupancy of a shared 25 by 25 PCA grid. n is how many entries survived. The table is claude-sonnet-5, mean over five seeds.
| Setting | Method | Quality | n |
| M&A, Disney | Novelty | 2.40±0.12 | 85±1 |
| M&A, Disney | Novelty+Evo | 2.51±0.07 | 135±5 |
| M&A, Disney | QD-Rec | 2.91±0.05 | 214±2 |
| M&A, JPMorgan | Novelty | 2.88±0.04 | 95±3 |
| M&A, JPMorgan | Novelty+Evo | 2.54±0.08 | 108±4 |
| M&A, JPMorgan | QD-Rec | 3.04±0.04 | 183±5 |
| Papers, open-endedness | Novelty | 4.96±0.02 | 578±5 |
| Papers, open-endedness | QD-Rec | 5.32±0.02 | 730±8 |
| Marketing, 3D printer | Novelty | 6.81±0.04 | 96±4 |
| Marketing, 3D printer | QD-Rec | 7.05±0.05 | 164±2 |
| Goodreads, series/YA | Novelty | 4.17±0.01 | 222±5 |
| Goodreads, series/YA | QD-Rec | 4.15±0.02 | 244±3 |
With an LLM judge, QD-Rec posts the highest mean quality in most settings and is never significantly worse than the best variant. The biggest gain is M&A, and the absolute level stays low: Disney moves from 2.40 to 2.91, and the archive from 85 to 214. Higher quality does not come from keeping a smaller archive. On JPMorgan, adding the evolutionary exemplars alone hurts: Novelty+Evo scores 2.54, below Novelty at 2.88.
Semantic diversity is slightly lower on M&A and best or tied elsewhere. Coverage is often higher, but not in every cell: on the open-endedness profile with sonnet, Novelty+Evo reaches 0.47 and QD-Rec 0.44. The weaker proposer breaks the pattern. With haiku on that profile, Novelty+Evo scores 5.03±0.07 against QD-Rec at 4.87, while its archive falls to 247±31, against 468 for Novelty.
On Goodreads the three variants land on similar star ratings. For the series reader with sonnet, Novelty is 4.17 and QD-Rec is 4.15. Stars grade the book, not the memo, and a book is not drawn again, because its rating is fixed.
relevance-MMR and a quality-weighted DPP are compared only on Goodreads. No setting of the relevance-diversity weight matches any QD variant on stars. On the picture-book reader, seed 0, baselines sit around 3.68 to 3.71 and the QD variants around 4.12 to 4.22. The baselines never observe a rating. The gap is rating feedback and the archive together. A few diversity-heavy settings match QD-Rec's semantic diversity, and some exceed its coverage.
One run costs about $13 to $62. QD-Rec costs the same as Novelty+Evo, about 1.3 times Novelty. More than 90% of that gap is the proposer prompt, because three exemplars roughly triple its input. The bandit runs locally. Replacing the embedder with Qwen3-Embedding-0.6B leaves the quality and archive-size ordering unchanged on one profile per domain. Mean quality rises through the run. Diversity flattens after about 100 evaluations: admission follows the LLM novelty bit, not an embedding-distance cutoff, so later entries land inside a region the archive already spans.
QD-Rec takes the loop used for algorithm search, a frozen model as the variation operator and an archive as the population, and points it at recommendation. Items cannot be invented. The open part of the search is the reason. At a matched budget of 1,000 evaluations, the bandit plus three prior hypotheses moves both M&A quality and archive size. The generator is a single prompt. The outer loop is already a lever.
Tens of dollars per run fits an internal decision that needs several distinct cases, such as acquisition theses or targeting angles. Against the two ablations the gain is incremental, and it shows up in LLM scores. Human stars do not separate the variants.
The bandit is linear in fixed embeddings. A nonlinear contextual bandit is untested. So is a coding agent that reads the archive and writes its own selection rule. Parents are sampled uniformly. The inner generator is one prompt. The paper keeps the loop simple on purpose and lists those three extensions as future work.
Quality measures how convincing a case for the item is. The judge is instructed that a hypothesis arguing against the acquisition, or against targeting the person, must score low even when the reasoning is correct. The proposer is told to make the strongest plausible case. The best M&A mean in the table is 3.04. On the paper's own 0-10 guide, 3 to 4 is still a weak case, and 2 or below means there is no real case. The jump from 2.40 to 2.91 is real relative to the ablation. The absolute level is still low.
The only human scores are Goodreads stars. The pool is books that reader has already rated. Of 542,145 readers, eight meet the filters, and two are used. Changing the memo cannot change the star, so this domain tests whether high-rated books stay in the archive. It does not test whether the reason is any good. Marketing uses synthetic people and fictional products. The companies are real. No deal outcome is in the data.
The writeup allows many hypotheses per item. The runs do not produce them. The diversity bonus drives the chance of reselecting an archived item to about zero, and no finished QD-Rec archive repeats an item. What the old hypotheses actually do is carry an angle onto a different item. Archive size can keep growing while the embedding span does not. The MMR and DPP comparison is not a clean ablation of item selection, which the paper states: those baselines see none of the ratings the QD runs use. Significance is a paired t-test on five seeds, uncorrected for the number of cells.