DialFork Benchmark Uses 36,278 Dialogues to Catch Procedural Hallucinations
DialFork · hf · 2026-10-27
IDIAP and collaborators release DialFork, a benchmark for detecting procedural hallucinations in conversational agents — errors not just in facts but in process steps.
- Scale: 36,278 dialogues, 24 tasks, 12 domains, 3 evaluation tasks
- Core innovation: flow-aware metrics that measure dialogue-level process deviation rather than content similarity
- Open source: github.com/idiap/dialfork
More from Research
- DREAM symposium at NeurIPS 2026: agentic AI for disease biology, abstracts due Nov 20 — Pseudomanifold · 2026-10-27
- AI designs lipid nanoparticles to deliver RNA drugs into solid tumors — paraschopra · 2026-10-27
- Building an open benchmark to compare AI approaches to analog chip design — paraschopra · 2026-10-27
- Researcher building benchmarks to tell whether LLMs compute or recall answers — paraschopra · 2026-10-27
- AI-written Community Note debunks brain injury lens study: only 14 of 41 participants analyzed — NathanpmYoung · 2026-10-27
- Anthropic's Claude finds 2^o(n) 3-SAT algorithm, refuting the Exponential Time Hypothesis — rickasaurus · 2026-10-27