Gemma 4 Quantization: Recovering 96% Reasoning Performance via Precision Redistribution
devildip · reddit · 2026-08-15
A study demonstrates the dramatic impact of Tensor Level Quantization Allocation under extreme budget constraints.
Key Results:
- At a 3.3GB IQ2XXS limit, reasoning scores improved from 28.9 to 69.5 (+140.54%).
- The final model is only 24% the size of the BF16 source but retains 96.74% of its reasoning performance.
- 10 out of 11 evaluated categories outperformed the imatrix baseline, with only stability regressing.
Methodology:
- No post-training, LoRA, or pruning involved.
- Builds an imatrix from category-oriented corpora, measures damage, and redistributes precision at the tensor level under a fixed byte budget.
Capability Retention:
- Knowledge QA: 97.50%
- Context: 95.83%
- Instruction following: 81.25%
- Coding: 58.49%
Related event: Tensor-Level Quantization Keeps Gemma 4 Strong at Tiny Sizes(2 posts)→
More from Models
- Dev: Claude writes 'zero bugs' on healthcare apps, 10x token efficiency — repligate · 2026-10-16
- Shanghai AI Lab unveils Intern Lumina U2, a diffusion LLM unifying image, video and 3D tasks — Shanghai-AI-Laboratory · 2026-10-16
- This Week in Tech breaks down Gemini 4, OpenAI's Dots agents, and Anthropic's leaked IPO filing — LouMM · 2026-10-16
- Sentry CEO Reveals $8,972 Claude Bill as Opus 'Scope Creeps' Through Team Workloads — zeeg · 2026-10-16
- Opus 5.5 wins: Anthropic reclaims token and spend share from OpenAI in 6 days — FinanceYF5 · 2026-10-16
- will.deeb: asking why a minor lab benchmarked against a 2-month-old frontier model is usually a stupid question — willcb · 2026-10-16