tufazip: Transformer-Based Lossless Compression Squeezes 1GB Wikipedia to 105MB

aigclink · x · 2026-11-20

tufazip applies a Transformer to lossless compression (truly byte-exact, not regeneration), shrinking 1GB of Wikipedia text to 105MB — an 89.46% reduction on enwik9 (100.96MB compressed + 4.43MB decompressor). It works by using the model's next-token probability distribution with arithmetic coding. The catch: it took two NVIDIA B200 GPUs about 7.3 days to compress and 7.5 days to decompress, excluding offline training — so it's nowhere near practical yet.

Related event: Tufa Labs open-sources tufazip, pure neural lossless compression takes second on long-running benchmark(5 posts)→

Original post →

More from Research

Research channel →