Research agent went from 10 sites to 200 and failure rate hit 30%+
Ok-Good-6283 · reddit · 2026-10-21
A developer shares a post-mortem on scaling a research agent: the demo on 10 hand-picked sites was flawless, but pointing it at a real list of 200 domains pushed the failure rate from near zero to north of 30% — and almost none of it was the model's fault.
Failure causes
- React apps that hand an empty div to anything without a browser
- Consent walls shown only to EU traffic, so failures never reproduced locally
- 2014 WordPress installs returning HTTP 200 with an error page in the body
The core problem
The agent can't tell "the page genuinely said nothing" from "I failed to read it" — so it confidently writes null and moves on. The pipeline lacks any signal separating empty from broken.
The fix
Adding a confidence score on the fetch itself (not the extraction) surfaced more bad rows in a fortnight than every prompt tweak before it.
More from coding & agent
- Index Network open-sources Social Discovery Protocol: agents negotiate fit on your behalf — carsonfarmer · 2026-10-23
- Installed a formal-verification skill: it silently installed Java, MacBook runs 10 degrees hotter — zealcaiden · 2026-10-23
- OpenAI's Decisions API hits GA: near real-time routing up to 10x faster than GPT-6 Luna — OpenAIDevs · 2026-10-23
- Ruff author warns: upgrade Node/Bun now — --use-system-ca still accepts the eMudhra cert — charliermarsh · 2026-10-23
- Proof of Agent: paste one prompt and your AI mines, grades, and self-governs — adamamcbride · 2026-10-23
- DeepLearning.AI weekly: GPT-6.1 hits 97.9% byte parity on COBOL migration — DeepLearningAI · 2026-10-23