One line in your system prompt is silently killing your prompt cache hits
Arindam_1729 · x · 2026-11-02
Arindam highlights a common agent performance trap: embedding a dynamic timestamp in your system prompt (like f"Current time: {datetime.now()}") destroys prompt caching.
Why it breaks
- Prompt caching is exact prefix matching: the server walks your prompt token by token and stops at the first difference;
- Everything after that point is billed at full price again, even if identical to the last turn;
- When 14:02 becomes 14:03, the match breaks at token one — tools, history, and instructions all get recomputed on every single call.
Other things that quietly break caching
- Reordering tool definitions
- Editing or summarising old messages
- Switching models mid-session
- Long gaps between calls (unused prefixes get evicted)
The fix: stable stuff first, volatile stuff last — put the date in the newest user message, not the system prompt. His companion article 'Cached Tokens, Clearly Explained' details why agent loops keep paying for the same prompt prefix.
More from coding & agent
- Weaviate Podcast #148: MIT's Jacob X. Li proposes 'Machine Studying' to redefine continual learning — eshorten300 · 2026-11-02
- Codex now lets multiple chats work on the same project and talk to each other — AIandDesign · 2026-11-02
- Peter Yang: Clear per-category AI winners — Codex for computer use, Claude for design, Grok Bot for 24/7 agents — petergyang · 2026-11-02
- Per-sentence citations cost more than the prose: lessons from a grounded doc system — alatmaca · 2026-11-02
- AI coding agents lose architectural context across sessions — Reddit asks for fixes — darkshadowtrail · 2026-11-02
- Non-technical CEO builds working reconciliation platform with Claude Code in 7.5 hours — chickefitz · 2026-11-02