ALP encoding officially lands in Apache Parquet with zstd-level compression and faster decoding
mgill25 · x · 2026-09-23
Apache Parquet has officially added the ALP (Adaptive Lossless floating-Point) encoding, a lightweight scheme offering compression ratios similar to zstd with much faster decompression, random-access support, and GPU/SIMD-friendly decoding.
Best for: decimal values stored as FLOAT/DOUBLE — monetary data, geographic coordinates, and scientific measurements.
Not for: data with wide exponent ranges or many significant digits, such as vector embeddings, which should stick with PLAIN/BYTESTREAMSPLIT plus general-purpose compression.
The post explains why systems commonly store decimals as floats: Parquet's DECIMAL type requires declared precision upfront, while pandas infers float64 and NumPy has no decimal dtype at all. ALP fills this gap across the Parquet ecosystem.
More from Infra
- The Datacenter as a Computer: Revisiting Luiz André Barroso's Google Legacy — JeffDean · 2026-09-24
- AMD Shares Behind-the-Scenes Look at Building and Testing Helios Racks — wkmyrhang · 2026-09-24
- NVIDIA gifts Unsloth a DGX Station; team teases faster quantization and efficient RL support — danielhanchen · 2026-09-24
- NVIDIA Nemotron 3 Diarization on Baseten: 500+ real-time streams per RTX PRO 6000 — baseten · 2026-09-24
- Unsloth passes 500M downloads as local AI adoption outpaces its own predictions — danielhanchen · 2026-09-23
- MLSys 2027 opens call for papers with Song Han as PC Co-Chair — songhan_mit · 2026-09-23