Vercel Cut CDN P99 Metadata Lookup Latency by 91% Across 80M Route Decisions/sec
cramforce · x · 2026-09-11
Vercel engineering blog details cutting CDN metadata lookup P99 latency by 91%. Its CDN executes 80M+ routing instructions per second and must look up metadata to decide how to serve each path.
Approach: Previously metadata was fetched and cached one path at a time, but large deployments hold hundreds of thousands of paths and every deploy caused cache misses. The team switched to grouped batch fetches that populate the cache for many lookups, then tuned the batch size via production experiments since over-fetching has its own cost — deployments got faster too. The post mentions binary search over sorted JSONL and a nod to bloom filters.
Related event: Vercel handles 10M daily deploys, cuts CDN metadata P99 latency by 91%(2 posts)→
More from Infra
- AWS on agent prompt caching: explicit + implicit patterns to slash token costs — AWS ML Blog · 2026-10-27
- Thieves spoofing GPS steal $100M+ in data center cargo since June — Ars Technica AI · 2026-10-27
- Lam Research: weekly token usage up ~250x in 18 months — Beth_Kindig · 2026-10-27
- AWS shows how to build enterprise agentic retrieval on Bedrock with CDK — AWS ML Blog · 2026-10-27
- Encrypted GPT-2 speedrun challenges FHE inference: best is 3.031 s/token, goal sub-1s — tensorqt · 2026-10-27
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27