Quantization Hits Finnish Hard: Hands-On KLD Tests Across Qwen3.8 Quants
MarkoMarjamaa · reddit · 2026-10-15
A hands-on measurement of how Unsloth quantization affects Qwen3.8 Flash's Finnish vs English ability, using KLD and Top-1% over Project Lönnrot, Gutenberg, and wiki corpora.
- Finnish clearly suffers: even Q8 shows noticeable KLD degradation, while English/wiki Q8 behaves as expected — low-resource languages trained with less weight mass degrade first
- Some quantizers use private eval corpora since wiki is over-fit into n-grams; results are mixed (better wiki KLD, worse top-1%)
- The author's 'chimera' hybrid swaps Unsloth Q4 for a disk-resident Q8-level n-gram, landing between Q4 and Q5 with zero memory cost; n-gram size doesn't affect Finnish ability, and Q5 is already close to Q8
- Nostalgia for Gemma-4 31B's perfect Finnish, but Q38FN's coding is in another class
More from Infra
- torch-dist-bench latency plots: 9 collectives benchmarked on 8x B200 — StasBekman · 2026-10-15
- Stas Bekman open-sources torch-dist-bench: PyTorch-only NCCL collective benchmarking — StasBekman · 2026-10-15
- Lambda benchmarks Nemotron 3.5 Super VL: one NVFP4 Blackwell GPU out-throughputs two in BF16 — TheZachMueller · 2026-10-15
- Neocloud risk isn't demand — it's that most builders can't deliver at gigawatt scale — JOBhakdi · 2026-10-15
- On-device AI settles on small active-parameter models: phone-class agents at 1B and below — lmoroney · 2026-10-15
- PyTorch models can now deploy to Arm Cortex-M with ExecuTorch runtime under 20 KiB — PyTorch · 2026-10-15