foldz: lossless GGUF compressor saves 8.66% across 9 Hub models
Simpli-curious · reddit · 2026-11-13
A Reddit user released foldz, an open-source (Apache 2.0, single C file, no dependencies) lossless compressor purpose-built for GGUF model files.
How it works: Generic tools barely shrink quantized weights because the data looks random. foldz instead parses the GGUF structure, unpacks quantized blocks into their real values and scales, then codes them with adaptive probability models. For 4-bit values it folds the upper half of the range onto the lower half via XOR (3 bits per value), and since the fold-marker bits are highly predictable, they're compressed with probability models — that's where the savings come from. Q6K, Q80 and float tensors get dedicated context models; everything else uses a byte-level model. Decompression is byte-identical, so files still work with llama.cpp, Ollama and LM Studio.
Results: 9 public Q4KM models all passed round-trip verification: 16.7 GiB → 15.2 GiB overall (8.66% saved). Qwen3-4B shrank the most (12.94%); Llama-3.2-3B the least (7.55%).
Limitations: 19 MiB/s (measured on a phone), no inference speedup, IQ quants/Q2K/Q3K/Q40 gain little, no zstd/xz baselines yet, pre-1.0 format. The author developed and tested everything on a phone, with the C implementation written by AI; they're asking for tests on other quant types and decoder fuzzing.
More from Infra
- Cloudflare hosts Thinking Machines' inkling 2 small on Workers AI at day-0 speed — michellechen · 2026-11-14
- Liquid-Codec decoder ported to Apple Neural Engine, 6x faster at 316x real time — JosephJacks_ · 2026-11-14
- AMD MI355X with FlyDSL kernels beats Blackwell running MiniMax M3.1 in vLLM — AnushElangovan · 2026-11-14
- Tencent paper: pruning neurons by routed contribution keeps 91-98% MoE performance — rohanpaul_ai · 2026-11-14
- AMD's hrx-loom stack: agent-written kernels beat llama.cpp across 51 LLMs on Strix Halo — jfowers_amd · 2026-11-14
- Microsoft Cloud tops $63B as AI buildout keeps scaling, says Nadella — thedealdirector · 2026-11-14