Agent failures don't throw: a 7-year engineer on clean double-runs and zero accountability
ilien-dev · reddit · 2026-09-21
An engineer with seven years of experience, now working mostly with agents, argues that agent failures look nothing like script failures: agents "do the wrong thing competently" and hand you a plausible success summary. The most dangerous pattern is the "clean double" — a retry fires, work happens twice, both halves report success, and if the step moves money or calls a metered API you find out from the invoice. Key points:
- The first question isn't the stack, it's who can reconstruct what happened when an agent does something expensive. If the answer is "ask the model," that person doesn't exist — models almost always confirm their own output, and agreement is the default failure mode.
- "The AI got it wrong" is never a usable incident-report conclusion: the agent doesn't attend the customer call, refund anything, or sign the data-breach letter. Whatever the subscription costs, accountability is zero and transfers entirely to whoever deployed it.
- Internal agents with self-contained blast radius are fine for learning; paid agents touching other people's money or data that you can only diagnose by asking the model are, in the author's view, indefensible — and a lot of current launches are exactly that.
More from AGI Musings
- Why people want to believe AI is fake: a "marketplace of rationalizations" — xuanalogue · 2026-09-21
- AI Assistants Cut Effort Well, but Not Friction, Argues Essay — _AustinCalvert_ · 2026-09-21
- Author Admits: On Most Things, I Don't Want AI to Optimize My Life — _AustinCalvert_ · 2026-09-21
- AI Panic Traced to Someone With Six Weeks of Work Experience, Critics Quip — kevinnbass · 2026-09-21
- Manus AI Said to Run 50 Social Media Accounts Autonomously, Sparking 'Dead Internet' Debate — CurieuxExplorer · 2026-09-21
- Naming Debate: 'Machine-Native Model' Proposed as Alternative to System One — aparnadhinak · 2026-09-21