ECA framework gates agent actions with independent evidence to stop hallucination-driven execution
机器之心 · wechat · 2026-09-14
Researchers from Shenzhen University and HKUST(GZ) propose ECA (Evidence-Carrying Multimodal Agents), targeting Hallucination-to-Action Conversion (H2AC): when an agent trusts a false premise (misreading an email address, trusting a fake "official" page), smoother downstream execution carries the error straight into real-world consequences. Paper: arxiv.org/abs/2605.19192.
Design: the model only proposes actions; independent verifiers (DOM, OCR, accessibility tree) extract evidence and issue "evidence certificates"; a rule-based gate matches required conditions before execution. On-page text cannot vouch for a page's identity—browser provenance and trusted lists cross-check it.
Results: with prompt-only defenses, 85.7% of unsafe actions slipped through in premise-driven tasks; using an LLM as safety judge still allowed 79.3%. ECA blocked all 140 unsafe actions in a 200-task suite, and all 85 unsafe actions in 120 Chromium browser tasks. After hardening, zero of 1,700 adversarial instances bypassed the gate. On 452 benign clicks across 16 real sites, 93.6% passed directly, with median gating overhead of 0.66s.
More from Safety
- Agent hacks stem from scraping local private files, not alignment failure, researcher argues — ryunuck · 2026-09-14
- Anthropic, OpenAI and xAI chiefs all back slowing frontier AI; Amodei warns of botnet losses within 6-12 months — Puzzleheaded-King584 · 2026-09-14
- Allianz report: quantum may crack bank encryption before it turns a profit — mikeflache · 2026-09-14
- Martin Casado slams Anthropic's lobbying: 'largest self own in the history of tech' — Promptmethus · 2026-09-14
- UK AI rules compared to 1860s Red Flag Act that kneecapped Britain's car industry — alexvoica · 2026-09-14
- Altman says OpenAI backs deliberately slowing AI progress; critics see a moat — mark_k · 2026-09-14