Cloudflare cut its AI code reviewer's inference bill by 68% with prompt caching and model routing
DanielLockyer · x · 2026-10-20
Cloudflare's postmortem on its AI code reviewer: with no architectural changes, average review cost climbed from $1.19 in April to $2.46 by late August. After auditing where tokens were going, the team cut average cost to $0.77 — down 68% from the $2.43 averaged over the two most expensive weeks — via reviewer code changes (40% of savings) and experimenting with model choice and effort levels (60%). Median review time also dropped from 4m09s to 1m32s.
Why costs rose: better models led engineers to produce larger, more complex changes, pushing more reviews into the most expensive "full review" tier (59% in March → 65% in August). Key levers included prompt caching and intelligent, risk-tiered model routing.
More from coding & agent
- 59 animations in a 43-min video: creator breaks down his AI agent workflow — MaxLenormand · 2026-10-20
- 4-char chunking slashes output latency: 10,000 words from 4.3s to 19ms — DanielLockyer · 2026-10-20
- OpenAI DevDay Tokyo demos two-voice agent-to-agent conversation setup — gabrielchua · 2026-10-20
- ClaudeST: an open-source native Claude client for Atari ST retro computers — Hakan_Ozalp · 2026-10-20
- OpenCode: the open-source Claude Code rival that runs GPT, Gemini or local models — thisdudelikesAI · 2026-10-20
- DevDay demo uses Decisions API to detect tool use inside live voice conversations — gabrielchua · 2026-10-20