New paper uses determinantal point processes to sample diverse yet high-quality LLM outputs

xtimv · x · 2026-11-21

A new paper, A Model of Diverse Sampling from Language Models, tackles a familiar failure mode: ask an LLM for 10 answers and you get 10 variations of the same idea. The authors revive determinantal point processes (DPPs) — distributions over sets of outputs that favor sets whose members are individually high-quality and whose embeddings point in different directions. A striking visualization samples code for a sorting-algorithm prompt, embeds each generation with a code language model, and projects to 3D via PCA: algorithm families like Quicksort, Merge, Insertion, and Heapsort emerge as distinct clusters. The paper includes experiments showing the method works.

Original post →

More from Research

Research channel →