Full FP8 n-gram table visibly improves local Qwen3.8-Flash-Next at zero speed cost
sillynoobhorse · reddit · 2026-10-17
A r/LocalLLaMA user found that common GGUF quants of Qwen3.8-Flash-Next heavily quantize the SSD-resident n-gram table (PLE), measurably hurting output quality. Packing the full FP8 PLE via Strata's plefp8pack.py and loading it with --ple-gguf costs only disk space and no speed, and visibly reduced dumb mistakes at IQ3S — confirmed with side-by-side HTML 3D scene outputs and by shipping new features to a KDE app via the Hermes agent.
More from Infra
- MiniCPM-V 4.7 runs locally at 100 tok/s using only ~3 GB of memory — NielsRogge · 2026-10-17
- Power-management chips: the invisible layer deciding AI data center energy efficiency — Olivier__OG · 2026-10-17
- MiniCPM-V 4.7 runs a 35B multimodal model in 3GB of memory, fully on-device — alifcoder · 2026-10-17
- Local Nemotron 3.5 on one A100 hits 169 tok/s, processes 100 clinical notes for 2.4 cents — MaziyarPanahi · 2026-10-17
- NVIDIA's open-source AIPerf benchmarking tool and Triton GPU tutorials for inference measurement — techNmak · 2026-10-17
- Official Triton GPU Programming Tutorials: Vector Add to Fused Attention — techNmak · 2026-10-17