A year of adversarial verification: 30% of confident agent outputs failed review

AIGODSEND · reddit · 2026-10-21

A team shares lessons from a year running an adversarial verification layer over their agent stack, with the core rule that the model writing an answer never grades it:

Reviewers drift: they over-reject risky-looking output, so the team ended up auditing the reviewers — every rejection tracked to a later verdict; reviewers with bad records lose weight, those who never approve get retired.
3 in 10 confident outputs died in review — wrong in ways a benchmark wouldn't catch but that cost money if shipped.
Three questions before shipping: what would falsify this? can state be restored if wrong? who audits the auditor?
Hardest part: pricing irreversibility before action; if no sequence restores the state, human labeling must happen before execution.
The bill is invisible: token spend is metered, failure cost isn't.

The takeaway: it's not about getting models to agree, it's about knowing what disagreement costs — and paying for the check before the failure.

Original post →

A year of adversarial verification: 30% of confident agent outputs failed review

More from coding & agent