Calibration-Free Quantization Method TQ Open-Sourced, Hits 92.4% Top-1 on Qwen 27B 4-bit
textclf · reddit · 2026-09-25
TextCLF released TQ, a calibration-free quantization method that needs no data, so models can be quantized the day they launch. Its 4-bit Qwen 3.8 27B shows mean KLD of 0.0282 and 92.4% top-1, close to calibration-based methods. The code is open-sourced as Quant Factory (GitHub), with quantized models on Hugging Face and a Docker image for vLLM serving. Currently 4-bit only, with 2-bit and 3-bit planned.
More from Infra
- Altman says OpenAI aims to replace Nvidia in the medium to long term — zephyr_z9 · 2026-11-11
- Chinese University Bought Nvidia H200 Chips Despite US Blacklist — pstAsiatech · 2026-11-11
- Baidu's ERNIE Evolve tops public bests on NanoGPT Speedrun, brings RSI to real inference optimization — xiaohu · 2026-11-11
- Baidu says RSI auto-optimized ERNIE inference, 6.24x decode speedup on Kunxin P800 — xiaohu · 2026-11-11
- Nvidia's supply commitments jump from $119B to $279B in one quarter, locking up 2027-28 memory — tengyanAI · 2026-11-11
- Train 27B models on a 24GB GPU: ternary optimizer cuts memory to 1 byte/weight — oguzeldereli · 2026-11-11