Agent claimed it fixed the security bug — the original PoC still fired: verify the trigger, not the diff
AIGODSEND · reddit · 2026-11-03
A Reddit post describes a classic coding-agent security failure: the agent closes a security ticket with a clean patch, passing tests, and a diff matching the advisory — yet the original PoC still fires on the patched build, because the agent fixed a different code path.
The author argues the failure isn't the model's reasoning but the check itself: we verified the diff, not the trigger. The check that would have caught it is simple:
- Freeze the PoC as a fixed fixture
- Apply the patch, then run the original PoC
- If the trigger has to change to reproduce, it's a different bug
The proposed workflow: every rejected action leaves behind a fixture — the exact trigger, the evidence the agent cited, and the check that would have caught it. The queue clears; the fixture stays. The post ends by asking readers what they freeze first when an agent says "fixed."
More from coding & agent
- DeepSeek publishes prompts for all 7 Harness Creator Mode plugin demos — aigclink · 2026-11-03
- DeepSeek Harness's Creator Mode generates plugins from plain language, threatening micro-SaaS tools — aigclink · 2026-11-03
- A one-night AI-built project nets 333 points and ~22k visitors — Aizkmusic · 2026-11-03
- Five manual checks before trusting an agent actually finished its job — Kodenstudio · 2026-11-03
- Open-source AI pit crew for iRacing runs 5 agents on Kimi, checks every number against telemetry — JeremyCMorgan · 2026-11-03
- Agent control plane benchmark: Smalltalk-80 cascades hit 100% compliance vs 2.5% for Gherkin — RandalSchwartz · 2026-11-03