Strix Halo one-command Docker for vLLM: Qwen3.8-Flash-Next hits 43 tok/s decode
asfbrz96 · reddit · 2026-10-31
The libstrix author shipped a one-command Docker image (vLLM fork + plugin + ROCm 10.1) for AMD Strix Halo, with tested compose recipes for Qwen3.8-Flash-Next, Qwen3.6-35B-A3B, Qwen3.8-27B, LFM2.5-24B-A2B and gemma-4 variants, plus a memory table showing weights vs KV cache at 131k context. Qwen3.8-Flash-Next required streaming its 102 GB n-gram table from disk and new kernels (int4 MoE GEMM, QSA sparse attention 1.5x Triton, QSA indexer). Real-text results: 1.6k tok/s prefill and 40-43 tok/s decode, up from 18, with KLD 0.008-0.019, 97-98% top-1 agreement and GSM8K 93 vs 92. Note: recent main-branch users should pull — int8 decode copies were over-enabled and could OOM non-target models.
More from coding & agent
- Web Monitors launches: track competitors, funding and new AI models via MCP — gaganghotra_ · 2026-10-31
- Inside Decision Models: How Perplexity's Decider Works and When to Use One — antoine_chaffin · 2026-10-31
- Epic launches open.epic MCP to connect dev agents to FHIR docs and API specs — DesireeCachette · 2026-10-31
- Open Ontologies: open-source 'Terraform for knowledge graphs' traces 901 consequences of one-line edits — anselm · 2026-10-31
- Supabase launches Explorer and Notebooks to unify database and log queries — dshukertjr · 2026-10-31
- Evals in 2026 101: one taxonomy for LLM apps, agents, and training/inference systems — WolframRvnwlf · 2026-10-31