Strata on 16GB VRAM: slow prefill at 162 tok/s as auto-tuning arms get skipped for memory
ckplscz · reddit · 2026-10-28
A Reddit user running Strata with Qwen3.8-Flash-Next Q20 on a 9060 XT 16GB + 32GB DDR4 Windows setup reports very slow prompt processing (162 tok/s pp4k vs 45.5 tok/s decode). Strata's auto-tuning JSON shows hostcorelast and kv-grow arms skipped due to insufficient free memory (25.6 of 28.4 GiB needed), and specminp / auxcpus experiments failing to win. Backend is ROCm/HIP. The post illustrates memory bottlenecks when running 64K-context models on 16GB cards.
More from Infra
- Cloud Providers Can't Forget What Enterprises Actually Run On — DavidLinthicum · 2026-10-28
- HUMAIN details site and power groundwork behind its AI infrastructure buildout — HUMAIN · 2026-10-28
- llama.cpp PR fuses MoE matmuls and GLU, boosting Qwen3 CUDA throughput up to 17% — jacek2023 · 2026-10-28
- A 5-minute visual explainer of what happens when you call an LLM API — HowDevelop · 2026-10-28
- GenieX brings any GGUF model to Snapdragon's Hexagon NPU, with real benchmarks — SkyFeistyLlama8 · 2026-10-28
- Cached tokens explained: how prefix caching cuts agent costs to 2% and what breaks the hit rate — Arindam_1729 · 2026-10-28