Multi-agent RAG at 18-25s latency: production lessons and the micro-agent architecture that fixed it
Odd-Relation-4587 · reddit · 2026-09-05
A production postmortem on enterprise multi-agent RAG: demos querying 50 PDFs feel instant, but with hybrid search (pgvector + Elastic), legacy SQL databases, and parallel tool calls, latency hit 18-25 seconds per query.
What broke and the fixes
- The "smart agent" antipattern: one agent with 10+ tools confused the LLM and quadrupled context length. They split into micro-agents: one router, two specialized retrievers, one synthesis agent
- Context bloat: raw vector results are no longer dumped into context — a strict ranking/dedup layer in Redis runs before synthesis
- Unmonitored tool calls: sequential blocking API calls moved to async Celery background tasks with Redis locks to prevent duplicate DB hits
Engineering reality: the agent logic took a week; caching layers, async queues, connection pooling, and CI/CD for non-deterministic code took nearly two months, with external engineers brought in for infrastructure. Takeaway: agentic AI is mostly high-concurrency software engineering disguised as machine learning.
More from coding & agent
- Vercel closed 1,500 Next.js issues in a month with a closability agent — Vjeux · 2026-09-05
- AgentMuxer launches public beta: an OpenRouter for agent capabilities with 30 providers — ycombinator · 2026-09-05
- Indie dev builds space game with hundreds of star systems using OpenAI's Astra, shares full workflow — Dimillian · 2026-09-05
- theo shares four hands-on ways to use Astra: slop audits, perf wins, and agent DX loops — lucasmeijer · 2026-09-05
- TIL: You Can Set a Token Limit on /goal in Codex — majidmanzarpour · 2026-09-05
- Meta's Muse Spark 1.3 hits #8 on Landing Page leaderboard, touted to reshape coding agent economics — yuwen_lu_ · 2026-09-05