Running Qwen Flash Next on a $1k RAM-rich GPU-poor box: 12tps but far better output
Positive-Stock6444 · reddit · 2026-08-30
A Reddit user shares benchmarks from running Qwen models locally on a 2018 ThinkStation P520 (Xeon W-2145, 256GB quad-channel DDR4, 12GB 3060, €1,000 total).
Qwen3.6 35B A3B Q4KM (former daily driver): 20GB RAM resident, 400tps prefill, 30-50tps generation, 128k context, and tolerant of other load on the box.
Qwen3.8 Flash Next UD-Q4KXL: 110GB resident, 200tps prefill, only 12-15tps generation, 65k context — but output quality is "night and day" better.
Key findings:
- Flash Next's performance collapses to 3-5tps under any memory contention; it's only really usable when opencode runs from another machine
- Synthetic random-content benchmarks gave completely wrong performance readings — realistic contexts are essential for measuring MoE models
- Full llama-server invocations for both models are included, covering GPU layer counts, CPU MoE allocation, quantized KV cache, MTP speculative decoding, and sampling params; Flash Next runs with unbounded thinking for full quality
Related event: Enthusiasts Test Local Qwen Flash Next: Memory Buys Quality but Speed Lags(2 posts)→
More from Infra
- Google Raises 2026 Capital Spending to Up to $205 Billion — emmanuelvivier · 2026-10-16
- Google open-sources AX, a Kubernetes-style orchestrator for billions of AI agent tasks — thisdudelikesAI · 2026-10-16
- Valtz v0.1.0.11 adds FLUX.3 support, local image gen up to 4K on Mac — TgoAI · 2026-10-16
- Baidu open-sources FlashMask: up to 1.55x faster than FlashAttention-4 for long-context training — BaiduResearch · 2026-10-16
- GYRO quant of Qwen3.8-Flash-Next tested: near Q3 quality at much smaller size — PlasticRevenue4601 · 2026-10-16
- RL train-sample systems are frontier AI's hardest infra problem — and a trade secret — brianryhuang · 2026-10-16