Vercel Cut CDN P99 Metadata Lookup Latency by 91% Across 80M Route Decisions/sec
cramforce · x · 2026-09-11
Vercel engineering blog details cutting CDN metadata lookup P99 latency by 91%. Its CDN executes 80M+ routing instructions per second and must look up metadata to decide how to serve each path.
Approach: Previously metadata was fetched and cached one path at a time, but large deployments hold hundreds of thousands of paths and every deploy caused cache misses. The team switched to grouped batch fetches that populate the cache for many lookups, then tuned the batch size via production experiments since over-fetching has its own cost — deployments got faster too. The post mentions binary search over sorted JSONL and a nod to bloom filters.
More from Infra
- Dev shares training dashboard: ~$11/b tokens cost with 'insane' MFU — jon_durbin · 2026-09-11
- Positron AI Raises $875M Series C at $5B Valuation, Deploying 50+ Atlas Racks at Oracle Cloud — Scobleizer · 2026-09-11
- Positron AI raises $230M Series B at over $1B valuation with Arm backing — seanmcdonaldxyz · 2026-09-11
- Cerebras Fast Inference Flips Agent Workflows: Fewer Parallel Agents, Same Output — MatthewBerman · 2026-09-11
- Baseten acquires Blaxel to build integrated cloud infrastructure for AI agents — baseten · 2026-09-11
- Skild AI's S1 learns robot tasks from one video, hits $100M revenue run rate — NVIDIA Blog · 2026-09-11