Task-aware quantization hits 99% of BF16 reasoning at 15% size, beating Unsloth by up to 19 points
devildip · reddit · 2026-09-08
Developer ByteOtter released TAK (Task Aware Knapsack), a task-aware quantization pipeline using task-corpus imatrix plus tensor-level precision allocation — no pruning, fine-tuning, or merging. On reasoning benchmarks it substantially beats byte-matched Unsloth quants:
- Qwen3.8-27B: 82.81% vs 77.34% Unsloth (+5.47); BF16 is 83.59%, so 99% of full performance at 15% size
- Qwen3.5-4B: 73.44% vs 61.72% (+11.72)
- Gemma 4 E4B: 69.53% vs 55.47% (+14.06)
- Gemma 3 4B QAT: 54.69% vs 35.16% (+19.53)
TAK builds an imatrix from a task-specific corpus, then promotes/demotes tensors within a byte budget to produce purpose-built quants. All tested on held-out data, across dense, QAT, and MoE architectures. Models are on Hugging Face; coding and math domains are next.
More from Research
- Proto-tools, a universal infrastructure layer for generative biology, now open source — jmschreiber91 · 2026-10-26
- New paper finds AI agents negotiate badly under uncertainty and break deals on purpose — xuanalogue · 2026-10-26
- MCP fault injection study: agent task scores drop 60% under realistic tool failures, scale doesn't help — dair_ai · 2026-10-26
- Study: AI reviewing AI misses most reward hacking; best models miss 25%+ of hacks — ronbodkin · 2026-10-26
- FLUX.3 Klein step-distilled to 4-8 steps with CFG baked in — Fit-Structure-6489 · 2026-10-26
- NLI alone rejects 5/6 new agent memories; a 0.45 similarity gate fixes it all — Alternative_Pin9598 · 2026-10-26