Ollama and Together Share Hard-Won Lessons on Keeping Coding Agents Fast

zainhas · x · 2026-11-17

zainhas (Together) and parthsareen (Ollama) wrote up lessons on keeping coding agents like Claude Code, Codex, and Pi fast with prefix caching. Since agents resend the entire conversation each turn, caching keeps long sessions fast — but only if both the harness and the inference engine do their part.

Key findings: a single timestamp in the system prompt can break the cache; changing reasoning effort can wipe the whole cache; and telling the agent its output was cut off will get it to read what it missed.

Original post →

More from coding & agent

coding & agent channel →