Entropy Distribution as a Fingerprint for Hallucinations in Generative Models
Mattia J. Villani, Pranav Deshpande, Akshay Seshadri, Romina Yalovetzky, Niraj Kumar
cs.AI
2026-05-27
JPMorgan casts hallucination detection as a test against a reference entropy CDF. Single-pass CES joins the top clique of much costlier multi-sample detectors across 80 setups.
Entropy is a familiar hallucination cue, but standard scores crush a generation into one number: perplexity, length-normalized entropy, or even length itself. Multi-sample methods work better and cost several extra forwards. White-box probes need hidden states. The JPMorganChase Global Technology Applied Research group argues for a finer claim: hallucination changes the shape and tail of the token-entropy distribution, not only its mean. After subtracting the mean, the shape difference stays significant in all 80 model–dataset experiments.
An oracle (a human, or GPT-4.1-nano as judge) labels a calibration set. Token entropies from non-hallucinated generations are pooled into a reference CDF F̂₀. A new entropy sequence is flagged if it does not look like draws from that CDF. CES takes two scalars, mean entropy and max entropy, pushes each through F̂₀, and returns their geometric mean, a score in [0, 1] that does not need per-task retuning. Without labels, the CDF is estimated on all calibration samples; if the hallucination fraction is γ, KS contamination is bounded by ε + γ.
Theory extends the DKW inequality to random-length vectors for uniform convergence of F̂₀, and shows that under mean and upper-tail shifts, CES true-positive rate goes to 1 while false positives decay exponentially in length. The working assumption is that, given length, non-hallucinated token entropies are approximately i.i.d. Appendix lag-1 autocorrelations say that is only approximately true.
Ten models (Falcon-7B, Llama-2/3/3.2, Mistral-7B, GPT-4.1 family, GPT-4o-mini) times eight QA sets (BioASQ, CoQA, DROP, GSM8K, NQ-Open, SQuAD, SVAMP, TriviaQA), 500 generations each, 128 or 256 new tokens. Median hallucination rate 0.27, from 0.04 on TriviaQA and CoQA to 0.78 on SVAMP and NQ-Open.
72 of 80 two-sample KS tests are significant at α = 0.05, median dKS = 0.100, median Cohen's d = 0.192. After mean-centering, 80/80 shape tests remain significant. Unsupervised CES wins 854 of 1279 pairwise comparisons against 16 baselines (66.8%), and beats 12 of 16 methods more than half the time. Gains include +0.085 AUROC versus generation length (85% win rate), +0.034 versus SelfCheckGPT (80%), and a 72.2% win rate versus semantic entropy. It loses to the KLE family and to embedding regression (43.8% wins, Δ = −0.017). Friedman χ² = 272.80; unsupervised CES has mean rank 6.29, inside the critical difference 2.779 of the best rank 6.16, statistically tied with KLE, semantic entropy, and Eigenscore at far lower cost. The unsupervised variant beats supervised CES in 78.8% of pairwise tests.
Black-box logits and one forward pass get close to multi-sample semantic detectors, with finite-sample calibration guarantees. In production the CDF can be built offline; online scoring is a mean and a max. Hallucination's definition stays with the calibration oracle, so the detector is not welded to one factuality taxonomy.
Reported AUROCs are in-sample rankings: the same 500 items build the CDF and score detection, with no held-out split, so the numbers are optimistic. Conditional i.i.d. is not strictly true. Generations shorter than about 10 tokens starve any distributional test. If the model was trained on wrong facts and is consistently wrong, dKS(F₀, F₁) can be zero and no entropy test has power. Experiments use greedy decoding on short QA; stochastic decoding and long-form generation are open. Some APIs hide even top-k logits. Judge quality caps the detector. That the unsupervised variant often beats the supervised one is also a warning about labels and protocol.