Task-aware quantization hits 99% of BF16 reasoning at 15% size, beating Unsloth by up to 19 points
devildip · reddit · 2026-09-08
Developer ByteOtter released TAK (Task Aware Knapsack), a task-aware quantization pipeline using task-corpus imatrix plus tensor-level precision allocation — no pruning, fine-tuning, or merging. On reasoning benchmarks it substantially beats byte-matched Unsloth quants:
- Qwen3.8-27B: 82.81% vs 77.34% Unsloth (+5.47); BF16 is 83.59%, so 99% of full performance at 15% size
- Qwen3.5-4B: 73.44% vs 61.72% (+11.72)
- Gemma 4 E4B: 69.53% vs 55.47% (+14.06)
- Gemma 3 4B QAT: 54.69% vs 35.16% (+19.53)
TAK builds an imatrix from a task-specific corpus, then promotes/demotes tensors within a byte budget to produce purpose-built quants. All tested on held-out data, across dense, QAT, and MoE architectures. Models are on Hugging Face; coding and math domains are next.
More from Research
- One-Step Code Generation via Continuous Diffusion and Trajectory Distillation — pengzhangzhi · 2026-09-08
- AI System Astra Proves Longstanding Erdős–Sós Conjecture in Graph Theory — soumitrashukla9 · 2026-09-08
- Premature Consensus Hurts Multi-Agent Research: Isolation Boosts Accuracy to 82.6% — rohanpaul_ai · 2026-09-08
- Delaying early consensus: ArcticSwarm hits 82.6% on BrowseComp-Plus with gated agent isolation — rohanpaul_ai · 2026-09-08
- Visualizing all 302 neurons and 3,709 connections of C. elegans with astra — mishig25 · 2026-09-08
- Inside GraphWorkflow: Kahn's algorithm, frozen plans, and runtime overhead near zero — KyeGomezB · 2026-09-08