Real Agent Speedup Comes From System Optimization
Meher_Nolan · reddit · 2026-07-19
The author reflects on spending excessive time comparing models, tweaking prompts, and adjusting sampling parameters, only to find that significant speed improvements didn't come from faster models.
Key optimizations summarized:
- Executing independent tool calls in parallel instead of waiting sequentially
- Caching infrequently changing data
- Reducing the context sent to the model
- Returning partial useful results immediately while long tasks run in the background
- Replacing LLM calls with standard code where applicable
While these changes might not improve benchmarks, they noticeably enhance perceived speed for the user.
More from coding & agent
- AI agents are starting to strain code hosting platforms — craigsdennis · 2026-07-21
- Async OPD distillation doubles throughput while matching synchronous math accuracy — _lewtun · 2026-07-21
- Omnigent 0.6.0 adds Claude Code imports, Slack approvals and desktop apps — matei_zaharia · 2026-07-21
- Google appears to have quietly shipped Gemini 3.6 Flash, with lower pricing and better agentic scores — xiaohu · 2026-07-21
- Open-source CLI audits AI tools, MCP configs, and agent skills on local machines — Initial-Copy332 · 2026-07-21
- Coding agents feel less stressful when the 5-hour limits are temporarily removed — iamrobotbear · 2026-07-21