Local LLM Speed Benchmarks on RTX 4080+3080: Qwen Flash-Next Prefills Faster on One GPU Than Two

Icaruswept · reddit · 2026-11-17

A Reddit user published decode/prefill speed benchmarks for seven low-quant local LLMs on an RTX 4080 Super + RTX 3080 setup running the PolyStrata stack:

The author argues the old "below Q4 is garbage" rule no longer holds for newer models: the Swift finetune of Qwen at Q2 is fast and excellent for data extraction, search, and running scripts; IQ3 codes roughly at Haiku level; GLM/DeepSeek Q2 variants work well as Claude subagents. Real task testing beats benchmark deviation from base when judging quantized models.

Original post →

More from Infra

Infra channel →