Cloudflare saved another 100TB of RAM by reworking consistent hashing in Rust
Cloudflare Blog · rss · 2026-09-19
- Cloudflare engineers traced unexpectedly high memory usage in their internal load balancer, Pingora Backend Router, to the pingora-ketama consistent hashing library.
- The post walks through how consistent hashing maps servers and tasks onto a shared number line, and why hash randomness makes per-server ranges uneven: with 100 servers, the coefficient of variation is 99%.
- Using 160 hash points per server (the NGINX/Pingora default) drops the variation to 8%, and the ketama algorithm allows weighting allocation by disk capacity.
- Optimizing the algorithm's math and Rust implementation reclaimed over 100TB of RAM globally, on top of 100TB saved by the DNS team last month.
More from Infra
- lateinteraction: nobody writes RISC by hand — compilers abstract away chip bifurcation — lateinteraction · 2026-09-19
- AI Infra Summit: Penguin Solutions and Astera Labs Bet Big on CXL Memory Expansion — BenBajarin · 2026-09-19
- Jev seen as local-model stand-in for low-latency apps; open-weight RLCD models expected — HankYeomans · 2026-09-19
- Hands-On: Running On-Device VLM Inference on Arduino Ventuno Q's Hexagon NPU — HowDevelop · 2026-09-19
- GPU price hike hits even the 1080ti, as local LLM token-speed numbers circulate — HankYeomans · 2026-09-19
- Bonsai 2 27B quantized beats Gemma 4 12B and Qwen 3.5 9B in 7GB 3D generation test — Fun-Meaning-6474 · 2026-09-19