Unifying the Definition of LLM Hallucinations
机器之心 · wechat · 2026-07-19
This article introduces an ICML 2026 position paper on the "unified definition of large model hallucinations," along with its accompanying benchmark, HalluWorld.
The core argument is that the "reference world" for hallucinations varies across tasks: summarization relies on source document fidelity, open-domain QA relies on factuality, while RAG and Agent tasks must explicitly consider retrieved documents, parametric memory, environmental states, and conflict strategies. Thus, the authors formalize hallucination as: under a specified reference world model, view function, and conflict strategy, an observable proposition in the model's output is false.
The article further stresses that not all errors should be classified as hallucinations: planning errors, refusals to answer, and instruction-following failures should be distinguished from actual hallucinations. Based on this framework, HalluWorld constructs three types of environments: GridWorlds, Chess, and TerminalTasks, using rule-based judgments instead of LLM judges. Experiments show that while frontier models are nearing saturation on statically visible information, they remain significantly weak in multi-step state maintenance, causal simulation, and knowing when to say "I don't know" under uncertainty.
More from Research
- PNAS paper shows a tiny billiard-ball system is a universal computer — undecidability lives in two dimensions — eigensteve · 2026-09-11
- New paper: Absolute pose estimation from affine cues and gravity direction — ducha_aiki · 2026-09-11
- LoMa Paper Ships REALLY HardPairs Dataset, Accepted at ECCV 2026 — ducha_aiki · 2026-09-11
- Johns Hopkins Launches Full-Stack Hands-on Robot Learning Class with SO-101 Arm Kits — _krishna_murthy · 2026-09-11
- SyncWorld: In-Context Robot World Model Simulates Unseen Views and Embodiments Zero-Shot — ChongZzZhang · 2026-09-11
- A 3D Pose Dataset for Dogs Released — ducha_aiki · 2026-09-11