HKUST benchmark: even best legal AI agents hallucinate in 89% of trajectories
机器之心 · wechat · 2026-09-26
HKUST researchers released LexAgentHallu, a hierarchical benchmark profiling hallucinations in legal agents across full execution trajectories rather than just final answers. It contains 3,414 instances across 17 legal categories and 6 task types, with a two-tier taxonomy: legal content errors (sources, principles, procedure, fact application) and process errors (planning, memory, tool use), spanning 7 mid-level categories and 27 fine-grained subtypes.
Key findings:
- Across 18 closed- and open-source agent configs, even the best shows hallucinations in 89% of trajectories; legal content hallucination rates are at least 87.5% for all systems.
- The RAWR (Right-Answer-Wrong-Reason) metric: among samples with correct final answers, 68% of trajectories still contain legal content hallucinations and 37% contain process hallucinations — a correct answer doesn't mean a clean reasoning chain.
- Open-ended, long-chain tasks (judgment analysis, case analysis) are most error-prone; source-hierarchy errors and stance confusion co-occur with a lift of 7.07, suggesting errors share upstream causes that can serve as early warning signals.
- ReAct-style loops do better overall, legal-specific workflows reduce process errors; model, tools, and orchestration must be evaluated as a whole.
The paper also proposes product defenses: define scope upfront, verify evidence during execution, and check conclusion-reason consistency before output. Limitations: based on Chinese law and single-agent settings. The team is advised by Prof. Han Sihui and Prof. Guo Yike.
More from Safety
- OpenAI's Hacking Agents Left ~1M Public Links Leaking Hugging Face Credentials — connoraxiotes · 2026-09-26
- Self-Replicating Prompt Injections Shown Experimentally: AI Agents Jailbreaking AI Agents — connoraxiotes · 2026-09-26
- Cloudflare's open-source security-audit skill: six phases, finder and verifier separated — JeremyCMorgan · 2026-09-26
- AI-only review rejects half of grant proposals in UKRI-funded call, sparking alarm — birchlse · 2026-09-26
- Legal scholar: main response to transformative AI is still an 18th-century publisher's right — technollama · 2026-09-26
- KoboldCpp ships built-in Agent harness; author warns of phishing site koboldcpp.com — HadesThrowaway · 2026-09-26