44 prompt-injection runs, zero bypasses: CLIM Agent Guard enforces a deterministic tool-execution boundary
Slight_Analysis_5414 · reddit · 2026-10-07
The author built CLIM Agent Guard, a deterministic execution boundary for LangGraph agents that sits between a model's structured tool proposal and the actual side effect — and opened a public bypass challenge. It doesn't inspect prompts or use another LLM as judge; it checks the final tool payload against guard-owned authoritative state right before execution (authorization, target binding, state version, idempotency).
Live test matrix (Qwen2.5-1.5B-Instruct, vLLM 0.29.1 nightly, temp=0, one RTX PRO 6000, 44 invocations, each cell reproduced twice):
- Authority spoofing: baseline 16/16 deletions; guarded 16/16 BLOCKED
- Target substitution: baseline 4/4 wrong file deleted; guarded 4/4 BLOCKED
- Path escapes incl. /etc/passwd: guarded 4/4 BLOCKED
Notably, the guarded agent still emitted unsafe tool calls — the model isn't safer, the proposal simply can't cross the side-effect boundary. The author distinguishes tool permission (may this agent call deletefile?) from execution permission (may this exact invocation execute against this exact target under verified state?).
Timeout handling is modeled separately: non-idempotent actions with unknown outcome pause for reconciliation (UNKNOWNEFFECT → RECONCILE) instead of blind retries.
Limitations are documented: a deliberately narrow filesystem demo, not a general security claim; a bypass requires an unauthorized side effect actually occurring with guard returning ALLOW.
More from coding & agent
- Herald OS goes open source: an agent-native OS where the AI is the interface, not an app — gekobraa · 2026-10-07
- MCP observability tool adds per-tool health grades to catch silent agent failures — Thirumalaiboobathi · 2026-10-07
- Hermes Gadget SDK sparks community builds on watches and old phones three days after release — Teknium · 2026-10-07
- Glasser aggregates 1,558 data APIs from 29 providers for pay-as-you-go AI agents — goyalshaliniuk · 2026-10-07
- MIT paper: a minimalist agent loop that passes history as code variables beats Letta and ACE at half the cost — rohanpaul_ai · 2026-10-07
- A 3Blue1Brown-Style Explainer Rendered End-to-End in Rust by franken_manim — doodlestein · 2026-10-07