Empirical Quantization Tests Reveal FFN Precision Dictates Quality Loss in LLMs
enginetown · reddit · 2026-07-31
A developer conducted full combined-model KLD quantization tests on the Qwen3.6-27B-Fable-Fusion-711 model. Unlike previous isolated component tests, this round built and evaluated the actual quantized model, fixing real degradations along the way.
Key Findings:
- Higher Compression Tolerance: This fine-tuned model tolerates much deeper compression than the base Qwen3.6-27B. The tolerance is wildly uneven across components; several attention and state tensors survived the most aggressive settings with zero measurable breakage.
- Tool Calling Stability Flipped: While tool calling usually breaks first in quantization, general purpose output degraded first this time, while tool calling held up remarkably well.
- FFN Precision is the Bottleneck: The most counterintuitive finding was that aggressive attention cuts cost almost nothing in file size if FFN precision is protected. Almost all real quality cost lives in FFN precision, not attention.
The author released three final builds (ranging from 3.90 to 3.33 BPW) where no category crossed the degradation red line. They also transparently listed remaining test blind spots (like stress testing math and tool calling), leaving the final risk assessment to specific user workloads.
More from Research
- Analyzing Quasar's Decentralized Training: Independent MoE Experts Bypass Communication Bottlenecks — markjeffrey · 2026-07-31
- WaveSight: Using Millimeter Waves for Non-Visual 3D Reconstruction — ctjlewis · 2026-07-31
- Expressing LLM Emotions via State Vectors: Context Pressure as Fatigue — Dependent-Advance468 · 2026-07-31
- Killing Hollow Success: Engineering Agent Verifiers and Drift Detection — Federal-Teaching2800 · 2026-07-31
- Living-Harness: Evolving Agent Frameworks by Turning Failures into Reusable Memory — burny_tech · 2026-07-31
- Relation Therapeutics expands GSK AI drug discovery deal worth up to $110M — anshulkundaje · 2026-07-31