Qwen3.8 Flash Next IQ3_S runs locally on 16GB VRAM at 250 tok/s generation, 5,600 tok/s prefill
yehyakar · reddit · 2026-10-25
Reddit user yehyakar shares detailed benchmarks running the Qwen3.8-Flash-Next IQ3S quant locally via the Strata/Basalt engine on an RTX 5080 (16GB), Core Ultra 9 285K and 62GB DDR5-6800, with 262K context and int8 KV cache.
- Agent coding session (opencode, 103 requests, up to 130K context): 5,011 tok/s prompt processing, 153 tok/s average generation (262 peak), 15.7GB VRAM used.
- Context sweep (fresh 15K–207K prompts): prefill peak 5,662 tok/s, 155 tok/s generation.
- 99.5% of prompt tokens hit the prompt cache; median time-to-first-token was 0.2s.
The takeaway: a 16GB GPU plus fast DDR5 can already sustain long-context local agent coding at usable speeds, with full numbers published for reproduction.
More from Infra
- NextDC plunges 40% in a day: the real AI bottleneck is the power line, not contracts — ccerrato147 · 2026-10-25
- A neocloud is a mortgage on a data center: rent comes later, interest is due now — ccerrato147 · 2026-10-25
- Korean AI data center firm Elice falls 32% on debut despite 400x oversubscription — ccerrato147 · 2026-10-25
- DeepSeek 552B model runs on a single AMD MI325X: 4 agents at 100 tok/s — realmrfakename · 2026-10-25
- AI's social license is written in the utility tariff: who pays when forecasts fail? — krishnan · 2026-10-25
- Amazon's ATTUNE coding agent lifts LLM training throughput 1.7x on H100/H200/B200 clusters — kalyan_kpl · 2026-10-25