Task-aware quantization hits 99% of BF16 reasoning at 15% size, beating Unsloth by up to 19 points
devildip · reddit · 2026-09-08
Developer ByteOtter released TAK (Task Aware Knapsack), a task-aware quantization pipeline using task-corpus imatrix plus tensor-level precision allocation — no pruning, fine-tuning, or merging. On reasoning benchmarks it substantially beats byte-matched Unsloth quants:
- Qwen3.8-27B: 82.81% vs 77.34% Unsloth (+5.47); BF16 is 83.59%, so 99% of full performance at 15% size
- Qwen3.5-4B: 73.44% vs 61.72% (+11.72)
- Gemma 4 E4B: 69.53% vs 55.47% (+14.06)
- Gemma 3 4B QAT: 54.69% vs 35.16% (+19.53)
TAK builds an imatrix from a task-specific corpus, then promotes/demotes tensors within a byte budget to produce purpose-built quants. All tested on held-out data, across dense, QAT, and MoE architectures. Models are on Hugging Face; coding and math domains are next.
More from Research
- Stanford Data-Centric AI Workshop reopens site, extends poster deadline — jure · 2026-10-26
- 'Vibe verification': ~60% of AI-written formal specs prove the wrong property — basedjensen · 2026-10-26
- Ito: a browser that detects browsing intent and auto-organizes tabs into Flows — andrewkuznet · 2026-10-26
- Cornell study says brains process language as linear sequences, challenging Chomsky's hierarchy — anirbanbandyo · 2026-10-26
- Hi3D V1.0 open-sourced with 2M training assets and 90% slicing success rate — kimmonismus · 2026-10-26
- Ortet evaluates 34 open-weight models on 50 health tasks, finds general leaderboards insufficient — elmanmansimov · 2026-10-26