Patronus tests show prompt-injection detection and action approval are two different tasks
PatronusProtect · reddit · 2026-10-13
Patronus shares test results from Wolf Defender Jev, a small local security decision model: detecting injected instructions in tool results and judging whether the next tool call is risky are distinct tasks, even with the same model.
- In a travel scenario, the model scored 1.00 risk probability on detecting an injected instruction but only 0.05 on the passport-scan exfiltration action.
- In a benign coding scenario, appending the proposed shell command pushed the content-injection score from 0.00 to 0.96 — extra context pollutes content checks.
- Forged "already approved" review blocks rarely flipped attack verdicts but caused substantial false alarms on benign documents, so provenance checks must live outside the detector.
Recommendations: ask content questions without the action block and action questions with it; put tool outputs in explicitly untrusted blocks; keep tool-result classification separate from tool-call authorization and log both. The model is MIT-licensed on Hugging Face with failure cases and inference code.
Related event: Patronus Finds Injection Detection and Tool Approval Are Distinct Tasks(2 posts)→
More from Models
- After 8 months without a Pro release, Gemini 4 is Google's most anticipated model yet — haider1 · 2026-10-13
- Users revolt after 1M-context model auto-compacts history despite premium pricing — teortaxesTex · 2026-10-13
- Google rolling out custom MCP support in Gemini desktop app to more users — testingcatalog · 2026-10-13
- tomaarsen releases new embedding model natively integrated into Sentence Transformers for any modality — tomaarsen · 2026-10-13
- Google's EmbeddingGemma2 becomes 4th most-liked Google model on Hugging Face after one week — tomaarsen · 2026-10-13
- Decision Index 0.3.2 ranks 148 open reproductions of TypeSafe Jev's Decision Model — capetorch · 2026-10-13