Ollama and Together Share Hard-Won Lessons on Keeping Coding Agents Fast
zainhas · x · 2026-11-17
zainhas (Together) and parthsareen (Ollama) wrote up lessons on keeping coding agents like Claude Code, Codex, and Pi fast with prefix caching. Since agents resend the entire conversation each turn, caching keeps long sessions fast — but only if both the harness and the inference engine do their part.
Key findings: a single timestamp in the system prompt can break the cache; changing reasoning effort can wipe the whole cache; and telling the agent its output was cut off will get it to read what it missed.
More from coding & agent
- MCP server turns your saved reels and posts into searchable notes for Claude — PaRtHuS · 2026-11-17
- Stardock CEO shows agents building games across PCs and Macs, with hard-won engineering lessons — draginol · 2026-11-17
- sandbox-mcp: Hanselman's MCP server gives AI agents disposable real Windows 11 machines — unixterminal · 2026-11-17
- ProceduralTerrains: open-source Three.js GPU terrain generator hits 2.0 — anselm · 2026-11-17
- Dev open-sources inference-optimisation agent skill he says made him $100K — Kyrannio · 2026-11-17
- Open-source Hermes Agent profile: cheap main model plus low-cost side-task pinning — alexcovo_eth · 2026-11-17