Increasing cache time saves 171M queries and 800GB egress daily
DanielLockyer · x · 2026-09-01
A developer reported significant performance gains by adjusting the cache staleness time from 5 seconds to 10 seconds. This strategy ensures requests are served from cache while refreshing in the background, eliminating 171 million queries and 800GB of daily egress traffic.
More from Infra
- Call for agent infra: Who will build the open source Codex-style browser? — hwchase17 · 2026-09-01
- Qwen2.5-72B Local Benchmark: 35 vs 65 Tokens/s Configs Analyzed — LittleCelebration412 · 2026-09-01
- Why is NVIDIA MIG still limited to 7 instances on B300/Rubin? — StasBekman · 2026-09-01
- Polymarket: 69% Chance Any State Enacts Data Center Moratorium by 2026 — Polymarket · 2026-09-01
- NVIDIA Groq 3 LPX hits 3,431 tokens/s output at 100K context in benchmarks — scaling01 · 2026-09-01
- NVLink adoption deepens investment in NVIDIA hardware — BenBajarin · 2026-09-01