Local LLM Speed Benchmarks on RTX 4080+3080: Qwen Flash-Next Prefills Faster on One GPU Than Two
Icaruswept · reddit · 2026-11-17
A Reddit user published decode/prefill speed benchmarks for seven low-quant local LLMs on an RTX 4080 Super + RTX 3080 setup running the PolyStrata stack:
- LFM2.5-24B-A2B UD-Q4KXL is the fastest: 250 tok/s decode and up to 7,930 tok/s prefill with both GPUs
- MiniCPM-V 4.7 35B-A3B hits 165-177 tok/s decode
- Qwen3.8-Flash-Next shows an anomaly: single 4080 prefill (1,070 tok/s) beats the dual-GPU config (679 tok/s)
- GLM 5.3 Flash and DeepSeek V4 Flash at IQ2 quants still manage 7-18 tok/s decode
The author argues the old "below Q4 is garbage" rule no longer holds for newer models: the Swift finetune of Qwen at Q2 is fast and excellent for data extraction, search, and running scripts; IQ3 codes roughly at Haiku level; GLM/DeepSeek Q2 variants work well as Claude subagents. Real task testing beats benchmark deviation from base when judging quantized models.
More from Infra
- Power Is a Red Herring for AI: Frontier Compute Efficiency Grows 10x+ Per Year — teortaxesTex · 2026-11-17
- Topos Bio teams with NVIDIA cuPotential to boost molecular dynamics throughput up to 3.6x — AllThingsApx · 2026-11-17
- Infrathon Benchmark Tests LLM Agents on Real-World AI Infra Optimization; Claude Opus 5.5 Leads — RulinShao · 2026-11-17
- PyTorch Day Japan 2026 set for Dec 10 in Tokyo: edge deployment and kernels — PyTorch · 2026-11-17
- New BIS rule forces restricted-region chip designers to win BIS approval, ending foundry transistor-count verification — terryyuezhuo · 2026-11-17
- Baseten hits ~2,500 tok/s on DeepSeek v4.1 Flash with Vera Rubin, 1.7x faster than GB300 — baseten · 2026-11-17