How to actually tell if an agent run got cheaper: a prompt-caching test design
Crescitaly · reddit · 2026-09-27
OpenAI's docs say reusable prompt prefixes on GPT-5.6+ stay cache-eligible for at least 30 minutes, and changing tool definitions or their order can break prefix reuse; cache reads are cheaper but writes cost too.
The author proposes a measurement design: run the same tasks with a fixed acceptance criterion, logging cached input tokens, cache-write tokens, total spend and latency per accepted result; compare stable tool definitions vs rebuilding the tool list each turn; use cache-miss diagnostics instead of inferring from dashboard dips.
Caveat: a high hit rate can still hide more tool calls, retries or worse answers. The experiment hasn't been run; it's a proposed framework with a question to practitioners.
More from coding & agent
- Open-source Codefolio adds multiplayer rooms, realtime editing and Claude/Codex AI integration — jasonkneen · 2026-09-27
- Opus 5.5 one-shots a Rocket League clone in Three.js — justalexoki · 2026-09-27
- Dad finishes a month of work in 5 minutes with Codex and Claude Code — yacineMTB · 2026-09-27
- Instructor RFC proposes typed decision models with Pydantic and OpenRouter — jxnlco · 2026-09-27
- GPT-OSS chat template bug silently drops past answers, degrading multi-turn coherence — arbv · 2026-09-27
- Matt Pocock uses AI animatics — AI stills + TTS — to pre-visualize course videos before filming — mattpocockuk · 2026-09-27