Unifying the Definition of LLM Hallucinations
机器之心 · wechat · 2026-07-19
This article introduces an ICML 2026 position paper on the "unified definition of large model hallucinations," along with its accompanying benchmark, HalluWorld.
The core argument is that the "reference world" for hallucinations varies across tasks: summarization relies on source document fidelity, open-domain QA relies on factuality, while RAG and Agent tasks must explicitly consider retrieved documents, parametric memory, environmental states, and conflict strategies. Thus, the authors formalize hallucination as: under a specified reference world model, view function, and conflict strategy, an observable proposition in the model's output is false.
The article further stresses that not all errors should be classified as hallucinations: planning errors, refusals to answer, and instruction-following failures should be distinguished from actual hallucinations. Based on this framework, HalluWorld constructs three types of environments: GridWorlds, Chess, and TerminalTasks, using rule-based judgments instead of LLM judges. Experiments show that while frontier models are nearing saturation on statically visible information, they remain significantly weak in multi-step state maintenance, causal simulation, and knowing when to say "I don't know" under uncertainty.
More from Research
- Structural ensembles beat single predictions in TCR:pMHC generalization study — quaidmorris · 2026-07-22
- Structural ensembles, not single predictions, drive robust TCR:pMHC generalization — quaidmorris · 2026-07-22
- enFoldX turns AlphaFold3 ensemble noise into a TCR–peptide–MHC predictor — quaidmorris · 2026-07-22
- RSS launches under OMSF to push structural biology data modeling at scale — MoAlQuraishi · 2026-07-22
- enFoldX tops 8 neoantigen scans and an unseen-peptide benchmark — quaidmorris · 2026-07-22
- enFoldX reaches AUC 0.82 on human VDJdb and transfers to mouse at 0.76 — quaidmorris · 2026-07-22