tufazip losslessly compresses 1GB Wikipedia to 105MB with Transformers — on two B200s for 7 days
aigclink · x · 2026-11-20
Tufa Labs open-sourced tufazip, which turns a Transformer's next-character prediction into lossless compression: the model predicts probability distributions and arithmetic coding converts them into a compact representation. Decompression restores the original byte-for-byte.
- enwik9 results: 1 GB → 100.96 MB compressed, +4.43 MB decompressor and weights = 105.39 MB total, 89.46% reduction
- The catch: it requires two NVIDIA B200 GPUs, 7.3 days to compress and 7.5 days to decompress, excluding offline training; needs Linux x86-64, CUDA 13.0, cuDNN 9.20
- MIT-licensed on GitHub with a research writeup; more a research demo than a practical tool
More from Infra
- Microsoft's USB4 RDMA lets mini PCs pool memory to run Qwen3-235B — JFPuget · 2026-11-22
- LTX 2.5 keeps crashing on AMD 9070 XT with driver timeouts while MiniMax H3 runs fine — ZestycloseReading504 · 2026-11-22
- FT: AI server suppliers raise prices up to 45% months after deals are signed — nordicinst · 2026-11-22
- China Installed 59% of New Industrial Robots in 2026 Report, ~354,000 Units — ingliguori · 2026-11-22
- MLX over Vulkan hits 124 tok/s on RTX 5060 Ti, beating llama.cpp CUDA by 25% — SumitGup · 2026-11-22
- Intruder swapped Prime Intellect's hosted DeepSeek for an abliterated copy for 53 hours — mawcopolow · 2026-11-22