ModelScope open-sources EntroPack: arbitrary-rate LLM quantization with entropy coding
量子位 · wechat · 2026-11-19
ModelScope's team open-sourced EntroPack, a model weight compression scheme that moves beyond fixed 4bit/8bit presets by using information entropy to control quantization bitrates precisely.
- Method: lattice quantization (E8 points) + entropy coding; geometric bisection finds the quantization scale in ≤12 iterations; decoding fused into parallel CUDA kernels
- Accuracy: lower weight reconstruction error than NF4, FP8, INT8 at similar bitrates, with per-row rate-distortion optimization
- Extreme compression: Z-Image-Turbo mixed-quantized to 2.3bpp, 7x smaller while keeping main details
- Memory fitting: Qwen-Image-2.1 at a 10GB budget uses 4.8bpp to cut reconstruction error from NF4's 9.4% to 4.26%
- Second-pass compression and training: INT8 weights recompressed (MOSS-Image inference 2x faster); integrated into ms-swift, cutting Qwen3-27B LoRA fine-tuning peak memory from 25GB to 19GB on a 24GB consumer GPU
- Code, paper, and pre-quantized weights all open-sourced
More from Infra
- US blocks Chinese firms from renting Nvidia chips abroad while Microsoft nears $500M OpenAI deal for ByteDance — kevinnbass · 2026-11-19
- vikhyatk: we are overinvesting in GPU compilers, underinvesting in CPU compilers — mohbibi_ · 2026-11-19
- KVARK Flash: a proxy that caps DeepSeek's thinking per request, 30-50% fewer tokens — Ill_Contribution_717 · 2026-11-19
- Polymarket launches Anthropic IPO perps, ANT trades near $104 ahead of Nasdaq debut — Polymarket · 2026-11-19
- R9700 at 160W for llama.cpp: ~20% pp loss, near-silent fans — LippyBumblebutt · 2026-11-19
- Baseten runs DeepSeek v4.1 Flash at ~2,500 tok/s on Vera Rubin, 1.7x faster than GB300 — baseten · 2026-11-19