Cache rereads are 94.8% of coding-agent tokens, and energy estimates differ 25x

Re-Reading Is Not Free: Token Traces and Cache-Aware Energy Estimation for LLM Agents in Software Engineering

Tina Vartziotis, Ketevan Peranidze, Rodopi Kosteli, Ezgi Sarikayak, George Dasoulas, Michael Keckeisen, Sotirios Kotsopoulos, Francesca Dominici

cs.SE, cs.AI

2026-11-16

Ten weeks of Claude Code logged 0.99 billion tokens, and 94.8% were cached rereads. Counting those rereads puts energy near 250 kWh, 25x the 10 kWh from output-only EcoLogits.

What problem this solves

Chat-style energy accounting does not describe a coding agent. EcoLogits charges GPU energy on output tokens alone and ignores input. Claude Code resends the whole conversation on every tool call. Prompt caching skips prefill on the repeated prefix, but decode still reads those cached keys and values. Parent logs keep only a subagent's last internal call.

The trace is one engineer on Claude Code from 9 June to 18 August 2026: six workstreams, 42 sessions, 752 task episodes, 992.5 million API-reported tokens. Models follow ordinary use, from Haiku 4.5 through Opus 5. Affiliations include TWT, Harvard, and MIT. Nothing was metered on the vendor's hardware.

Method

Each call reports four counts: uncached input, cache writes, cache reads, and output. Fresh tokens are the first two summed. Context is fresh tokens plus cache reads. Fresh tokens take a full prefill and a KV write. A cache hit skips prefill, and every new output token still reads the KV cache for the whole context. Rereading notes is cheaper than a first read. The cost is not zero. The transcript copies usage onto every content block, so summing records inflates tokens by about 2x; copies from resumed sessions add another 28%.

The hybrid estimator starts from the analytical GPU model of Vartziotis et al. and changes which tokens enter which term. Prefill runs on fresh tokens only. Attention reads run on the full context. Facility power, PUE, and embodied carbon follow EcoLogits. Three scenarios price the cache as free, as attention only (cache-aware), or as a fresh prefill. Every term is nondecreasing in its token counts, so the order is free, then attention-only, then fresh prefill, for any coefficients. Attention positions scale with output length times context, so the model is applied per request and summed. Coefficients come from Hopper microbenchmarks on an NVIDIA GH200. Memory factors were fit on dense 8B to 72B models and uncached requests of 500 in and 500 out, within 5% to 27%. Nothing was refit on this trace.

Hosted models use the EcoLogits registry's low and high active-parameter bounds. The headline figure is their geometric mean. KV width is 1024, the grouped-query size used for Llama 3. The original check covers short uncached requests only. Comparisons use the low bound. The boundary leaves out client-side tools, network transfer, training, and idle capacity.

Results

Main thread, 3,723 calls:

ClassTokensShare
Cache read940.5M94.8%
Cache write48.3M4.87%
Output3.62M0.36%
Uncached input0.168M0.017%

About 260 cached tokens are read per token generated. Uncached input plus output is 0.38% of the trace.

Context grows with the session. In 23 sessions of at least 20 turns, mean cache reads per turn rise from 70k to 372k, 5.3x. Longer sessions weigh more in that pool; equal session weights give 3.8x. In 86 episodes of at least 10 turns the rise is 1.4x. Output per turn is 0.9x across a session and 0.3x inside an episode.

Accounted workload, 3,829 requests, US grid. Central values are the geometric mean of the parameter bounds.

EstimatorEnergyCarbon
EcoLogits, output only9.89 kWh (7.24-13.5)3.97 kg
Hybrid, cache free36.8 kWh (18.5-73.4)15.0 kg
Hybrid, cache-aware248 kWh (105-585)96.2 kg
Hybrid, cache as fresh456 kWh (232-895)176 kg

248 kWh against 9.89 kWh is about 25x in energy and 24x in carbon. Treating the cache as free leaves 36.8 kWh, so charging rereads supplies a factor of about 6.7 and the rest of the model gap about 3.7. Pricing cache reads as fresh prefill is 1.8x the cache-aware figure, so prompt caching removes about 46% of the hybrid estimate. At the low bound the ratio is 14.5.

At that bound, attention reads are 90% of IT energy, about 79 kWh, 12x the EcoLogits total on the same basis (6.6 kWh). Prefill is about 6%. Per-call hybrid energy tracks context (Spearman 0.59) more than uncached input (0.34). At fixed output length the hybrid estimate spreads 14.3x on Sonnet; EcoLogits spreads 1.4x.

A call near the medians of context, fresh input, and output (Sonnet 4.6; 804 fresh, 153k cached, 389 out) is 0.71 Wh in EcoLogits and 11.5 Wh cache-aware. Those cached tokens cost 3.7 Wh of attention and would add 9.3 Wh if prefilled from scratch, both at the GPU low bound.

Hold the tokens fixed and swap the model. Opus costs 10.2x Haiku, and every workstream lands between 9.6x and 10.5x. At the low bound the same trace is 16.9 kWh on Haiku, 92.5 on Sonnet, 96 on GPT-5.4, 172 on Opus, and 1,042 on Gemini 2.5 Pro, against 105 as actually mixed. Gemini uses a 200B to 600B active-parameter guess outside the calibrated range, so the 62x span is not an efficiency ranking. Starting each episode in a new session cuts cached tokens by 76%. The central estimate falls from 248 kWh to 97 kWh (61% less) and carbon from 96 kg to 38 kg. EcoLogits does not move. Restoring dropped information would take extra calls, so 61% is an upper bound on the saving.

Subagents appear in 44 of 746 working episodes (5.9%). The 106 foreground delegations made 1,334 internal calls. Parent logs report 5.97 million tokens; the transcripts hold 59.8 million, a 10x undercount, median 7.4x. Energy does not scale by ten. Repricing each internal call on its recorded model moves those delegations from 4,793 Wh to 7,840 Wh, 1.6x, because the logged final call already carries the longest context and most inner calls ran on Haiku 4.5 while the parent log often inherited Sonnet or Opus. Eight recoverable background runs bring the low bound from 105 kWh to 110 kWh, about 4.8%. Delegated tokens are 0.6% of the log and at least 6.4% of the trace. Route those subagents to Sonnet or Opus and the cancellation goes away.

Median episode size runs from 166k tokens to 835k. Turn count, session position, and primary model explain 88.9% of variance in log episode tokens. The task label alone explains 6.3%. 46% of episodes call no tool, 73% of Understand episodes among them, yet Understand still sits at a median 182k tokens.

Why it matters

Without changing models, the large step on this trace is a new session or a compacted context between unrelated tasks. That cuts the hybrid estimate by about 60%, and the figure is already an upper bound. Model tier is the other lever: on the same tokens Opus is about 10x Haiku. That comparison is per call. A smaller model may need more turns to finish the task.

In household units, at the central value, the low bound is about 2.4x smaller. Ten weeks of this work is 0.19 kWh per working day and 43 kWh a year under EcoLogits, 0.4% of a US household, against 4.9 kWh a day and 1,070 kWh a year under the hybrid estimate, about 10%. A median request is 3x to 4x a published chatbot prompt (0.24 to 0.31 Wh) under EcoLogits and 60x to 77x under the hybrid model.

Neither number is a meter reading. Neither estimator has been checked on long cached contexts. Removing the attention-read term entirely still leaves a 1.4x gap. No assumption varied in the paper brings the two estimates together. The token mix is billing data.

Limitations

One engineer, one client. Directories were chosen after the fact at roughly a one-million-token cutoff, and the excluded volume was not recorded. Task labels are unvalidated keyword rules; Refactor has 7 episodes. Session length, task difficulty, and context compaction are not separated.

The absolute kilowatt-hours sit high, and the soft term is the one that carries about 90%. Its memory factors were fit with a dmodel-sized cache on 8B to 72B models. Here the width is dkv, and the upper active-parameter bound reaches 200B, where the product of those factors rises from about 15 at 72B to about 60. Switching the width back to dmodel multiplies the total by 6.7. Decode-time checks on Llama 3.3 70B, gpt-oss-20b, and Qwen3-8B (two H100s, batch 1) show time per output token growing nearly linearly with context, R-squared 0.98 to 1.00. Caching removes prefill time and does not remove the per-token cost. Charging attention at the 26.1 pJ/bit an H100 spends streaming memory at full power drops the low bound from 105 kWh to 33 or 98 kWh. The longest experimental context is about 93k tokens. The trace median is 153k.

The grid factor is the US annual average, 384 g/kWh. Serving region is unknown. The same central energy is 9.7 kg on the Swedish grid and 147 kg on Poland's. Counterfactuals freeze the recorded calls, so they do not say whether a cleared context or a smaller model still finishes the task. The authors expect cache-read dominance for other users of this client. Other assistants are untested. The subagent undercount depends on the client version.

Terms

Source

What people are saying

Related papers

All paper explainers