Your p99 latency benchmark may be lying: a deep dive into coordinated omission
Franc0Fernand0 · x · 2026-09-11
The author's second latency article explains why the most common benchmarking approach—waiting for each response before sending the next request—inadvertently coordinates with the system under test and hides real tail latency, a problem known as coordinated omission.
- Example: a 100ms GC pause would queue dozens of slow requests in a real workload, but the benchmark logs just one and moves on, so p99 looks great while lying
- Covers sources of latency variance from CPU caches to garbage collectors, and how latency compounds across services
- Includes a small ping tool to measure your own network's tail latency
More from Infra
- SemiAnalysis: OpenAI reportedly negotiating ~1GW TPU commitment while seeking sites — dylan522p · 2026-10-28
- I tested 11 video upscaling methods for AI video: local, Topaz and cloud compared — lopas99 · 2026-10-28
- Thermo-backed CTO says AI is in its dial-up phase, touts disaggregated compute for an ultrafast 'broadband moment' — beffjezos · 2026-10-28
- DeepSeek-V4.1-Flash Q2 runs at 41 tok/s on one RTX 3090, 90% token match vs API — musicurgy · 2026-10-28
- USearch v3 RC ships after 1,000 days, its libraries now embedded in 550K-star projects — viglovikov · 2026-10-28
- SK Hynix says memory demand will outpace supply through 2030 — Chemical_Hawk_6307 · 2026-10-28