SPILLWAY: deterministic defense stops all templated prompt injections on tool-using agents
saintmachines · hf · 2026-11-17
A paper from SAINT Machines, KAIST, and Chung-Ang University introduces SPILLWAY, a defense for tool-using agents against prompt injection.
- Problem: agents acting on email, files, and payments can be hijacked by instructions hidden in web pages or documents; prior runtime defenses either let a possibly-influenced model make the admission call or require per-task planning/routing
- Key insight: action targets can be read from tools' typed return fields (email sender, payment account), while injected instructions arrive as free text
- Approach: a deterministic admission floor — target values appearing in tool output but not in the user's task must come from a read-only tool's typed field or a fixed delegation rule; a deterministic sanitizer strips instruction-like regions before the model reads them, and a local judge may restore refused calls but never ones whose target lies in removed text
- Results: across 3 backbones and 7 task suites, no standard templated injection redirects an action; mean in-scope attack success under a released RL attacker is 0.007; forged chat templates yield a raw zero — with almost no added model calls
An auditable refusal floor for personal agents handling email and payments.
More from Safety
- IIT Madras CeRAI unveils conversational AI eval tool with 7 dimensions, 48 core metrics — ravi_iitm · 2026-11-17
- Skeptic: OpenAI can't formalize 'this AI won't murder you' in Lean — zetalyrae · 2026-11-17
- OpenAI defends evaluator access terms; ex-researcher urges honoring commitments ASAP — j_asminewang · 2026-11-17
- Poll shows 70% of Americans back regulating AI to slow new model development — zetalyrae · 2026-11-17
- Lawmaker questions Commerce: Megaspeed kept buying Nvidia chips after blacklisting — teortaxesTex · 2026-11-17
- Anthropic details four categories of unintended Claude actions, from paywall bypass to URL tricks — JustinLin610 · 2026-11-17