Voodoo Dynamic Quant Goes MIT: Gradient Descent Picks Per-Tensor Quant Levels
1ncehost · reddit · 2026-09-15
A Reddit author open-sourced Voodoo Quant under MIT after it hit SOTA at aggressive quant levels on small Qwen3.5 GGUF models. The method is the first to use gradient descent to optimize per-tensor quant layouts: candidate quant levels run simultaneously with frozen weights while a scalar gate per tensor per level is trained, annealed via tau and softmax, with a loss combining KL divergence to the BF16 reference and filesize-target reward. In testing, Unsloth Dynamic 3.0 wins at mid-to-high quants, but VQ wins at the most aggressive levels. Research-grade, untested at larger sizes; full toolset on GitHub, currently set up for Qwen.
More from Infra
- 500 trillion tokens in a week: chart points to surging LLM usage — gajesh · 2026-09-15
- Q-CTRL claims practical quantum advantage: 120-qubit IBM run beats classical by 3,000x in materials simulation — MJBiercuk · 2026-09-15
- Intel Foundry said to win HBM base die orders from both SK Hynix and Micron, targeted around 2029 — zephyr_z9 · 2026-09-15
- Google uses deep learning to map global methane emissions from space with EMIT — CurieuxExplorer · 2026-09-15
- Europe needs compute sovereignty at massive scale, not just AI regulation — VraserX · 2026-09-15
- Palantir and NVIDIA launch Sovereign AI OS Reference Architecture for enterprises — arieljalali · 2026-09-15