Building reliable agent harnesses: lessons from 4,000 runs of an open-source deep research agent
Public_Umpire_1099 · reddit · 2026-09-16
After 5 months and 4,000 runs, a developer shares reliability lessons from building what he calls the closest open-source equivalent of Manus/Perplexity: a deep research + builder agent whose reports match Gemini, Claude and ChatGPT, with a 300-consecutive-successful-builds benchmark (reset on any failure).
Key insight: a self-hosted Qwen3.6 27B that built a perfect site got stuck in infinite file-reading loops — not because the model was weak, but because the harness's edit tool demanded exact character matches, file reads silently truncated content, and the loop counted reading as progress. Fixes: forgiving edit matching, line-targeted operations, paginated reads, and a nudge when reads streak without changes. He also rants against community slopware (6k+ LOC files, no tests) and will release raw run data on GitHub.
More from coding & agent
- When an LLM plans an executable agent DAG, where do you draw the trust boundary? — Repulsive_Sugar_5252 · 2026-09-16
- "Never again": user rants Nemotron 3 Ultra can't find .github and stalls for 90 minutes — dwerps · 2026-09-16
- Personal AI's real test: does your judgment survive after you close the window? — sujingshen · 2026-09-16
- Coding agent so fears AWS costs it wrote a test banning NAT gateways — mimi10v3 · 2026-09-16
- Muse Code raises contributor tier limits and resets usage for all subscribers — giffmana · 2026-09-16
- OpenAI reportedly orchestrates 10k internal agents without quality loss, while users' sub-agents 'just burn tokens' — RexDouglass · 2026-09-16