Node.js async zstdCompress() was 23x slower due to missing pledged input size
AI Snake Oil · rss · 2026-09-28
Cloudflare engineer Cherry found that Node.js's async zlib.zstdCompress() ran 3-5x slower than the sync version when precompressing static assets at build time.
The root cause: zstdCompressSync() hands zstd the entire input in one call so it can infer the size, while the async version writes input and ends the frame in separate calls — zstd never learns the size and sizes its tables for an unbounded stream.
His PR (#66358) defaults pledgedSrcSize to the input's byte length, making async output byte-identical to sync. Benchmarks on 162 production files (up to 370KB each, Linux x64): level 9 went from 300ms to 113ms, level 12 from 4521ms to 194ms, level 19 from 9696ms to 1474ms. Windows shows the same pattern.
More from Infra
- AI's Power Paradox: Data Centers Strain Grid While AI Agents Optimize Energy — ingliguori · 2026-09-28
- Beyond 2-4 GPUs there are no off-the-shelf cases, says HF dev planning 3D-printed shrouds — TheZachMueller · 2026-09-28
- The biggest buildout in history: data centers are industrializing intelligence — Dr_Singularity · 2026-09-28
- Anthropic's Contracted Value Exceeds $180B, Per NVIDIA — zephyr_z9 · 2026-09-28
- David Friedberg: Open-Source Models Make Data Centers Unnecessary for 90% of AI Use — MarvinTBaumann · 2026-09-28
- Old-School Infra Engineers on PB-Scale AI Logs: Hadoop Did This on a Tuesday — basedjensen · 2026-09-28