pigzpp: C++ Rewrite Achieves 50x Faster Parallel Compression Library

jedisct1 · x · 2026-08-27

The paper introduces pigzpp, a from-scratch C++23 rewrite of the widely deployed pigz parallel gzip utility.

Design Overhaul: The original pigz relied on process-global mutable state, making it non-reentrant and unsuitable for direct embedding. pigzpp transforms this into a thread-safe library with a single accelerated DEFLATE core.
Multi-language Support: The core is exposed to C++, Python, WebAssembly, Go, and Rust. Built on this same core, it also provides application-level conveniences like multi-entry ZIP archives and a fast PNG codec.
Performance Gains:
The portable zlib-ng backend preserves gzip/zlib compression ratios.
The Intel ISA-L backend (x86-64 only) serves as a fast path, producing about 10% larger output at level 6. In benchmarks on a text corpus, zlib-ng achieves 2.1x the CLI throughput of pigz and 16x that of Python's in-memory gzip; ISA-L reaches 8.5x and 50x respectively.
Compatibility: Outputs remain standards-compliant and cross-decode with gzip, pigz, and unzip.

The result is a self-contained, multi-platform compression stack and a case study in AI-assisted modernization under automated compatibility tests.

More from Infra