Engineer's honest take: AI agents look great in demos but never pass the production bar
cclalex · reddit · 2026-09-12
A practitioner asks whether anyone has actually automated a manual workflow end to end with AI agents — document comes in, someone reads it, applies rules, makes a decision, writes to other systems — and argues everything they've built looks impressive but never meets the production bar.
Key pain points:
- Double work in QA: a labeled regression set never captures real-world nuance, so verifying every output means doing the job twice.
- Prompt-tweaking loop: constant tweaks break the agent in subtle ways, so you're never free of the QA cycle.
- Complexity trade-off: splitting into deterministic flows plus LLM calls is more robust but harder to change and needs engineering support.
The author solicits real automation examples and how teams built confidence in their agents.
More from coding & agent
- Reddit debate: agent memory, not the model, is the real bottleneck — Luvena21 · 2026-09-12
- You're treating Claude like a junior intern: 8 copy-paste prompts for senior-level AI collaboration — ahuja_priyank · 2026-09-12
- Decagon shares 19+ ablations on using GEPA for test-driven prompt optimization in production — kastnerkyle · 2026-09-12
- GeoGuessr as an RL env: 4B VLM trained with OpenEnv and TRL to play the game — SergioPaniego · 2026-09-12
- Chinese firm used Claude Code to run 20+ dating apps with 4,700 AI personas deceiving 25,000 users — luisdans · 2026-09-12
- Security researcher: cloud providers need defensive agents and rogue-GPU kill plans — kuza55 · 2026-09-12