Qwen3.8-27B Quantization Analysis: 4-bit Damage Peaks at Context Start

sadnessdevil · reddit · 2026-08-23

This post evaluates the KL Divergence (KLD) of various Qwen3.8-27B quantization checkpoints using code datasets on 8k and 32k contexts.

Key Findings:

• Huge Gap Between 8-bit and 4-bit: The best 4-bit score (0.00835) significantly outperforms the worst 8-bit (0.00071).
• High Variance in 4-bit: Models with identical sizes show KLD ranging from 0.01364 to 0.02976, indicating file size is a poor metric for 4-bit selection due to calibration/rounding differences.
• Group Size Impact: Checkpoints with group size 32 outperformed those with group size 128, even when the latter kept some modules in BF16.
• Damage Concentrated at Context Start: Surprisingly, all 24 models show much higher KL divergence in the first 500 tokens.
• Specific Layers Safe: Quantizing lmhead, embedtokens, and linearattn showed no performance impact in this test.

Original post →

More from Research

• Intelligence is proportional to data: reorder tasks by data volume and the jagged frontier becomes a spiral — sebkrier · 2026-10-16
• Doc2Harness: New benchmark finds best coding agent passes only 53% across 329 cases — _akhaliq · 2026-10-16
• Stanford's Fei-Fei Li team open-sources OpenWAM, a unified world-action model framework for robots — jiqizhixin · 2026-10-16
• NVIDIA's TacitVLA: robots reason without reading their own thoughts, 3.4x lower latency — arankomatsuzaki · 2026-10-16
• Cohere Labs world model course session 4: DreamerV1-V4 architectures in one go — Cohere_Labs · 2026-10-16
• GPT-6.1 Sol audit finds 6% of arXiv papers had false main theorems, down to 1.5% recently — nitarshan · 2026-10-16

Research channel →