IDEA: inference-time audit detects visual-trigger backdoors in GUI agents

huawei-noah · hf · 2026-11-19

HKUST, Peking University and Huawei Noah's Ark Lab propose IDEA, a step-level inference-time audit that detects visual-trigger backdoors in open-weight GUI agents. Backdoored agents pass clean acceptance tests but misbehave only when a screen trigger appears — a supply-chain risk existing detectors miss. IDEA compares instruction- and region-induced shifts in decision state against clean-step conformal baselines, flags the "intent-evidence dominance" signature when an unreferenced region hijacks a step, localizes triggers with controlled false positives, and can hand off to humans.

Original post →

More from Safety

Safety channel →