With 98% Cache Hits in Coding, 400-600 t/s Prefill Already Hits Diminishing Returns
nomorebuttsplz · reddit · 2026-09-03
A Reddit user analyzes which workloads are actually prefill-bound: in opencode coding sessions, cache hit rates are 98%, so 400-600 t/s prefill is sufficient—decode takes 75-90% of the time, and even instant prefill would boost throughput by at most 25%. He also notes low cache hit rate doesn't imply prefill bottleneck: a research agent run on GLM 5.3 with only 150 t/s prefill and single-digit cache hits still spends 90%+ of time decoding at just 15 t/s in max mode. He's asking what real workloads remain prefill-bound.
More from coding & agent
- LLM one-shots a Minecraft 3x3 piston door by building its own Linux test infra — Vjeux · 2026-10-18
- Dev builds Mumbai-inspired 3D manga background generator from OpenStreetMap data — CurieuxExplorer · 2026-10-18
- Solo founder builds a money-printing data service on Claude's Pi Durable harness — MicahBerkley · 2026-10-18
- Dev builds Arch-based OS with custom desktop and browser using Claude Code — Snoo-10493 · 2026-10-18
- Five SaaS agent workflows fail the same way — and it's never model intelligence — usehive · 2026-10-18
- Claude skill generates hand-drawn interactive diagrams that teach kids how things work — CatAstro_Piyush · 2026-10-18