Building a hallucination detector taught us "hallucination" isn't one category: partition, don't boolean
Top-Shopping539 · reddit · 2026-09-13
An AI engineer at Uniflun recounts iterating on a hallucination detector for a customer-facing order agent. Version one flagged replies matching confirmation patterns without a createorder call — but most flags were false positives: references to existing orders, conditional promises, and negation forms in a dialect where the affirmative pattern sits inside the negated form. Two independent problems hid in one composite metric, making every discussion of the number contestable.
The fix: change the output from boolean to a five-way partition by asking two questions in sequence — does the reply assert an order exists, and does tool evidence back it? (noclaim/valid/validstatusref = nobody's wrong; claimedbutfailed/phantom = agent bug; wrong bucket = evaluator bug, author's own.)
The most valuable bucket is claimedbutfailed: tool called but errored, agent confirmed anyway — identical to fabrication from the customer's side, yet invisible for months because the happy-path condition was satisfied.
Two high-leverage lessons: verify the identifier, not the phrasing — extract the order number and require it to appear in a real tool result, which beats any lexical work and fixes existing-order false positives for free. And beware predicates quietly absorbed into "fine" by composite metrics — the dashboard looked fine throughout.
More from coding & agent
- BountyScout bot hourly-scans GitHub bounties and floods maintainers with AI applications — gleech · 2026-09-13
- Cognition's SWE-2 impresses early user with solid coding, but lacks Codex-style computer use — CtrlAltDwayne · 2026-09-13
- Better AI 3D output costs more upfront but saves tokens and time vs. endless iteration — chaseleantj · 2026-09-13
- No prompt one-shots a polished 3D asset — human-judgment iteration took 17 more minutes — chaseleantj · 2026-09-13
- Workflow tip: generate a reference image first, then iterate your 3D output against it — chaseleantj · 2026-09-13
- Skill-based 3D generation: 68k to 153k tokens but far more realistic leaves — skills set expectations, not teach — chaseleantj · 2026-09-13