We hardened our agent against attacks — and accidentally broke it for real customers
vivvvekredddit1122 · reddit · 2026-11-15
A team tightened agent guardrails after finding policy-bypassing attacks; the attack success rate dropped, but support tickets surged as legitimate requests were refused because the agent grew suspicious of anything resembling an attack. Their fix: every security test case gets a matching legitimate case that should still succeed; each release tracks attacks blocked AND valid requests completed (if one rises as the other falls, that's not a win); refusals are reviewed like approvals, via full traces. An agent that refuses everything is perfectly safe and completely useless.
More from coding & agent
- Dev builds a full Git Tag manager UI with Copilot Canvas in VS Code — mariorod1 · 2026-11-15
- A playable tank game built with Claude Code: Opus plans, parallel Sonnets code in worktrees — IamHuggos · 2026-11-15
- AskEditor embeds a hand-editable video timeline inside Claude via MCP — MrxAnomynous · 2026-11-15
- He gave an AI coach GPS, smart-home access and a wallet — it now runs his winter training — Hopeful_Composer3977 · 2026-11-15
- Agent apologizes with a swear: '"fair" was the wrong word, I won't use it again' — cocktailpeanut · 2026-11-15
- User's Grok-powered bot phones Enterprise and wins a partial rental refund end-to-end — altryne · 2026-11-15