Quant War Escalates: Unsloth Quants Beat ISTA-DASLab's gsq-rco in Reproducible Benchmarks
TingTingin · reddit · 2026-11-07
A quant benchmark dispute is brewing in local LLM circles: ISTA-DASLab's gsq-rco quants face criticism from unsloth over unreproducible numbers and unpublished methodology. The basalt inference engine author switched from gsq-rco to same-size unsloth quants after testing — his charts show unsloth UD-IQ1M with KLD 0.2560 vs gsq-rco Q20's 0.4329 (-40.9%) and top-1 agreement 83.19% vs 78.24%, meaning unsloth quants stay closer to the BF16 reference.
More from Infra
- HyperFlow H3 Flash: 13.7B Few-Step Model Generates Video With Stereo Audio, Runs on RTX 5090 — Apprehensive_Sky892 · 2026-11-07
- JITEngine synthesizes custom LLM inference engines for Apple Silicon, claims up to 2.7x speedup — AccBalanced · 2026-11-07
- Dev predicts 2027's local inference boom: integrated hardware + OS + inference stack, not just GPUs — vincent_koc · 2026-11-07
- vLLM Office Hours Hits 59 Sessions: 76 Speakers from 24 Organizations and Counting — vllm_project · 2026-11-07
- Dev runs 284B-parameter DeepSeek-V4-Flash on CPU only, squeezed into 64GB DDR5 RAM — solyarisoftware · 2026-11-07
- Omnigraph: open-source graph database with git-style branching for multi-agent shared memory — solyarisoftware · 2026-11-07