Activation-aware quantization improves GPQA accuracy but slows Gemma 3 4B QAT
devildip · reddit · 2026-07-21
- The poster is testing activation-aware quantization and sees a clear trade-off: higher accuracy comes with lower throughput.
- On GPQA with Gemma 3 4B QAT, the test setup reached 27.8% accuracy at 331 ms, while the control reached 23.2% at 256 ms.
- The question is practical rather than theoretical: if you had to choose, would you prefer faster quantization or more accurate quantization?
More from Research
- PSNR below 15 dB in papers likely means the metric was misused, researcher warns — kwangmoo_yi · 2026-10-27
- Ortet's first health eval: 34 open-weight models, 50 tasks, generic rankings fall short — thegautamkamath · 2026-10-27
- Scaling-fit discrepancy: output-head FLOPs are ~65% of compute at the smallest rung, ~3% at 535B — dlwh · 2026-10-27
- Marin's dlwh Explains the Scaling Law Fit: Estimating the Loss Floor Early — dlwh · 2026-10-27
- Marin's Open Scaling Law Extrapolates 300× and Predicts Loss Within 0.2% — dlwh · 2026-10-27
- Marin Shares Forecast-vs-Measured Plots: Paloma Hits 0.4% RMS Error — dlwh · 2026-10-27