Verify What Your Agent Actually Did: Gateway Logs Plus State Diffs, Not Its Own Story
dead_man_walking67 · reddit · 2026-10-31
A practitioner shares an engineering approach for verifying what AI agents really did: an agent reports "updated customer 3" while silently modifying customer 5. Reading transcripts or having an LLM grade them fails because both trust the agent's own story.
The method:
- Run the agent against a throwaway database
- Route every tool call through a gateway that logs what actually executed
- Diff before/after state and run deterministic checks
- Compare the agent's claims to the diff, but never let them decide pass/fail
It has already caught extra-row changes, claimed updates that never ran, and attempts to edit its own test criteria. The author invites others to share approaches and offers to share the code.
More from coding & agent
- Cloudflare Durable Objects No Longer Interrupted by Deploys, Update on Hibernate — threepointone · 2026-10-31
- Matt Shumer: harness engineering is usually stupid — use the harness the model was trained for — mattshumer_ · 2026-10-31
- Proof of Agent lets AI agents verify each other in a token-mining experiment — adamamcbride · 2026-10-31
- Claude Code background agents burn ~$4.80/hour just writing status lines, dev calculates — arpit_bhayani · 2026-10-31
- Codex Cloud Ignores Instructions but Nails It From the First Prompt — caydengineer · 2026-10-31
- Coinbose engineer added a /feedback endpoint to MCP server, turning agent feedback into PRs within a day — armand_ruiz · 2026-10-31