Ling-3.0-flash Quantization Benchmarks: MoE Architecture Preserves Decode Speed
AcanthisittaOk1699 · reddit · 2026-08-12
A community developer shared benchmark results for the Ling-3.0-flash quantization ladder (124B total params, 5.1B active) on a single DGX Spark.
- Performance: Across the Q5KM to Q6K quantization ladder, single-stream decode speeds consistently fall between 32 and 40 tok/s. Q5KM is the fastest (40.2 tok/s) and near-lossless.
- MoE Advantage: Because the model only activates a tiny fraction of params per token, dropping bit width barely impacts decode speed. This is a stark contrast to dense models, where lower bit width usually trades off for throughput.
- Comparisons: For scale on the same hardware, DeepSeek V4 Flash measured 16.5 tok/s, making the Q5 Ling-3.0-flash roughly 2.4x faster.
Related event: Single DGX Spark Tests 124B Model with High-Speed Decoding(2 posts)→
More from Infra
- AI-Driven Timing-Aware Power Optimization Uses Learned Ranking With Baseline-Preserving Repair — Amit Sharma · 2026-10-16
- Nscale AI datacenter plagued by coolant leaks, transformer fire, 10 safety violations — saranormous · 2026-10-16
- WSL 3.0 ships with native containers and Linux 6.18 kernel, syscall overhead down ~16% — solyarisoftware · 2026-10-16
- Are 22 GB modded RTX 2080 Ti cards a viable budget local LLM option? — Dave86ch · 2026-10-16
- Intel's neural-compressor: SOTA low-bit LLM quantization with FP4/NVFP4 support — HaihaoShen · 2026-10-16
- Tencent squeezes 770B-parameter Hunyuan Hy4 Preview into 214 GiB at ~2.38 bits per weight — coolmum18 · 2026-10-16