How to Verify if an Agent Actually Did the Work
Slightly_F0ol1Sh · reddit · 2026-07-16
After running agents in production for months, the author's biggest headache isn't whether the model can talk, but how to confirm it actually got the job done.
Core Issues
- Agents often claim they "saved/found it" and report completion, but no real side effects actually occur.
- These errors don't surface immediately; they accumulate downstream and are only discovered much later.
Current Practices
- Stop trusting the agent's own output.
- Use a "dumb" method for external verification: e.g., checking if a new row actually exists in the database, or if a file was truly written to disk.
- If there are no real results, fail the step immediately rather than passing silently.
Unsolved Challenges
- Long-term drift is harder to catch: an agent might work fine today but silently fail weeks later without triggering system alerts.
- The author wants to hear how others distinguish between "the agent actually did it" and "it just claimed it did."
Related event: AI agents in production: don’t trust narration, verify outcomes(8 posts)→
More from coding & agent
- BELL: Open-Source Stack Combines Browserbase, Eve, iMessage and Stripe Link for Personal Agents — jeff_weinstein · 2026-10-27
- Two engineers rebuilt a FedEx supplier's billing core in 3.5 months vs an estimated 7-8 — alex_verem · 2026-10-27
- GPT-6.1 Terra gains ~20 points on Terminal-Bench 4.0, rivalling GPT-6.1 Sol — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra scores 48 on AA Coding Agent Index at one-third the cost of GPT-5.6 Terra — ArtificialAnlys · 2026-10-27
- Shift-1 router cuts coding agent spend up to 55% while keeping SWE-bench accuracy — ypatil125 · 2026-10-27
- Non-coder vibe-codes a Godot JRPG with Claude Code in two weeks, music and art synced — ADHDavid · 2026-10-27