HCPD: Zero-Source LLM Hallucination Detection Gains 10 Points AUROC Over Baselines
量子位 · wechat · 2026-08-29
An ICML 2026 paper, "Zero-source LLM Hallucination Detection with Human-like Criteria Probing," tackles the strictest setting: no access to the target model's probabilities, internal states, knowledge bases or reference answers—only the Q&A text itself.
HCPD mimics how human experts evaluate dynamically: from generic dimensions (factual correctness, logical consistency, etc.), an agent adaptively generates fine-grained criteria and weights per sample, scores each dimension, and outputs interpretable justifications. Training uses GRPO reward alignment with weak supervision from semantic similarity on a 1-10 scale; inference aggregates multiple parallel evaluations to reduce variance.
Evaluated on TriviaQA, SciQ, NQ-Open and CoQA across LLaMA and Qwen families (7 models): average AUROC of 88.19% on LLaMA-3.1-8B (+10.20 over the runner-up) and 89.62% on Qwen-3-8B (+10.15). Detection remains stable when the generating model changes, since it relies on shared natural-language space rather than model-internal signals.
More from Research
- Neural Network Basics: Train CNNs with PyTorch & TensorFlow — Al_Grigor · 2026-08-29
- Decision Trees & Ensembles: XGBoost Tuning and Feature Importance — Al_Grigor · 2026-08-29
- MIRIAD Benchmark Reproduction: Hybrid Search Hits 0.8476 NDCG@10 — philipvollet · 2026-08-29
- Proteus: Open-source framework lets AI agents rewrite their own harness code — aigclink · 2026-08-29
- Tsinghua Yao Class Instructor Mengdi Xu: Scaling Alone is Not Enough for General Robots — 量子位 · 2026-08-29
- Rewarding only final answers may lead models to ignore reasoning correctness — A_K_Nain · 2026-08-29