Open-source CQ 4-bit QLoRA platform beats Unsloth on val loss by 17.4%
textclf · reddit · 2026-10-28
A developer released Training Factory, an open-source platform for QLoRA fine-tuning of 4-bit quantized models, built around a self-developed calibration-free quantization method called CQ (Coded Quantization).
Benchmark highlights (Qwen 3.8 27B):
- Quant quality: CQ 4-bit mean KLD 0.0282 vs unsloth-bnb-4bit 0.0330 on Wikitext-2, with smaller disk footprint (18GB vs 22.4GB)
- QLoRA training: on Alpaca, single A40 48GB, matched hyperparameters — CQ final validation loss 1.0315 vs Unsloth NF4 1.2484 (17.4% lower), with similar peak VRAM
- Weak spot: speed (27.6s per optimizer step vs Unsloth 3.2s); author notes downstream quality not independently verified
The repo covers the full workflow: downloading a CQ 4-bit base, training a LoRA adapter, and serving it with vLLM.
More from Infra
- Palmer Luckey Unveils BlueChip, Billed as First American-Made Computer in Decades — whurley · 2026-10-28
- OpenRouter's entire DeepSeek traffic could run on a single GB300 rack, worth just $6-7M — teortaxesTex · 2026-10-28
- Open-source ai-phone: local LLM answers real calls on your Mac, nothing leaves the machine — Agent007_MI9 · 2026-10-28
- Agents built a custom inference engine: AMD MI350X serves GLM 5.3 FP8 37% faster than NVIDIA B200 — yisongyue · 2026-10-28
- Why RTX 5090s cost $10k: leaked factory footage shows cards converted into AI accelerators — Different_Fix_2217 · 2026-10-28
- Anthropic's S-1 admits its data centers rely on natural gas, exposing it to fuel costs — TobyWalsh · 2026-10-28