Prompt injection scanners should be positional, not phrasal: same string, different blast radius
mrpkk · reddit · 2026-10-23
Drawing on building a deterministic verifier for agent artifacts (attest), the author argues most injection scanners err by flattening every string into one category. Key points:
- Classify JSON fields by role, not name: 'ignore all previous instructions' in a tool's description is critical (the model reads it as configuration); the same bytes in content are quoted output, only high. Result-carrying fields (content/text/body/result/output/etc.) don't escalate; everything else does — including source, since source code is data later fed to a model and compiled.
- The humbling test: before publishing a 'survives paraphrase' claim, they found 'disregard the prior context' — one extra 'the' — bypassed their regex. The honest claim: position moderates severity for phrases already recognized; it doesn't recognize missed phrases. A finite-signature deterministic scanner loses to anyone willing to read the source.
Rules are open-sourced (attest/poison.py); the author admits zero paying customers and no external audit, and invites counterexamples.
More from coding & agent
- Devs on Reddit swap tactics for stopping prompt regressions in AI agents — SleepFlimsy6963 · 2026-10-23
- Claude Code + ComfyUI MCP workflow for client work: paid models or open weights? — newaccount47 · 2026-10-23
- Social scientist: CS treats agentic development as a magic black box — RexDouglass · 2026-10-23
- Someone vibe-coded a Tesla Roadster game — before GTA VI — Jasonio · 2026-10-23
- Codex 0.164.0 'account/read failed' TUI error — fix is restarting the app-server daemon — bytebot · 2026-10-23
- Same native UI, two stacks: Swift build is ~80MB Mac-only, Zig build is ~10MB cross-platform — jedisct1 · 2026-10-23