Power-limited 5090 + 96GB DDR5 hits 150-200 tok/s decode on Qwen3.8-Flash-Next locally
z0_o6 · reddit · 2026-10-02
A Reddit user reports running Strata on a power-limited RTX 5090 with 96GB of DDR5-6400, achieving 150-200 tok/s decode and 5-6k tok/s prefill on Qwen3.8-Flash-Next at IQ3S quantization with a 128k-token context (8-bit KV cache). It shows that with partial offloading to system RAM, local long-context inference can remain very usable.
Related event: Strata Lets Consumer PCs Run 180B Models Locally(3 posts)→
More from Infra
- Dev submits PR adding periodic checkpoints to GKE sandboxes so agents don't lose progress — yoimnotkesku · 2026-11-19
- AWS Lambda Adds GPU Support via Managed Instances for Inference and Media Workloads — DavidWells · 2026-11-19
- Nvidia Allegedly Made GPUs Artificially Scarce to Keep Prices Sky-High — GaryMarcus · 2026-11-19
- Leak Hints at Classified 1M-GPU Training Cluster in China That Skips HBM — teortaxesTex · 2026-11-19
- Tencent leaned on Japanese neocloud for AI compute; DeepSeek, Zhipu best positioned — teortaxesTex · 2026-11-19
- Open-source prefill engine for GLM-5.3-Flash beats vLLM by 3.2x on 4x B200 — CShorten30 · 2026-11-19