How Agent Backends Handle Long Loops
sandyyevans · reddit · 2026-07-14
The author encountered a classic engineering challenge while building an agent backend:
- The frontend handles session interactions, while the backend calls the LLM, executes tool calls, and advances tasks through multi-turn ReAct loops.
- User session histories are persisted in Postgres by sessionId, with multiple users sharing the same LLM client.
- The backend pushes real-time status updates (reasoning, tool calls, execution results) to the frontend via SSE.
The current bottleneck: a single user request can trigger multiple ReAct loops, taking significant time and easily causing concurrency build-up on the web service. The author is asking how the industry typically handles this; while switching to an async queue is an option, they worry it would break the ability to stream real-time status updates via SSE.
More from coding & agent
- Soft Clamp cuts tool-call overuse in multi-teacher distillation, from 13.7% to 9.0% — antgroup · 2026-07-21
- SpecJudge runs locally on Ollama to pick the right-sized AI model for your project — jokiruiz · 2026-07-21
- A developer maps out six design rules for CLIs that humans and AI agents can both use — yujiezha · 2026-07-21
- A coding-agent skill that forces ADHD-friendly, answer-first output — ayghri · 2026-07-21
- A set of agent skills for CAD, robotics, and hardware design — earthtojake · 2026-07-21
- Outlines keeps LLMs on-rails with structured outputs — dottxt-ai · 2026-07-21