Strix Halo-specific kernels speed up vLLM prefill by up to 31x at long context
asfbrz96 · reddit · 2026-10-29
A Reddit user released libstrix, hand-written HIP kernels for AMD's Strix Halo APU (gfx1151, Radeon 8060S, 128GB unified memory) packaged as a stock-vLLM plugin — no fork or patching needed.
Why it exists: stock ROCm kernels target MI300, so vLLM on Strix Halo was painfully slow: a 256k prompt on Qwen3.6-35B-A3B took over two hours.
How it works: installs via pip and hooks into vLLM's plugin system; only activates on ROCm + gfx1151. Each kernel handles one specific shape and falls back to vLLM's native kernel otherwise. Covers MoE GEMMs, attention prefill/decode, Qwen GDN layers, and int8/int4/MXFP4 linears.
Numbers (single stream, stock → libstrix):
- Qwen3.6-35B-A3B bf16: 64k prefill 260 → 2,617 tok/s (10x); 256k prefill drops from 7,568s to 241s (31x)
- LFM2.5-24B-A2B: 64k prefill 304 → 3,615 tok/s
- gemma-4-26B-A4B AWQ: 64k prefill 138 → 2,225 tok/s; MXFP4 gains are inflated by a weak emulation baseline
Accuracy: KLD vs. stock is at noise floor on Qwen models (0.001–0.006, 97–99% top-1 agreement, no GSM8K difference); Gemma-4-26B drifts more (KLD 0.2). Limitations: decode gains are model-dependent, gfx1151-only, and the build must match vLLM's torch/ROCm versions.
Repo: github.com/asf0/libstrix
More from coding & agent
- Agent sandbox boat.dev sees record growth day with 100k-view demo and nonstop Stripe orders — zack_overflow · 2026-10-29
- Dev picks Cursor's $60/mo Grok plan as best budget coding combo — pramodk73 · 2026-10-29
- Claude Duet research preview lands in users' hands as Anthropic quietly expands access — RileyRalmuto · 2026-10-29
- LangChain Core 1.7.1 Ships Two Fixes for Streams and Mermaid Diagrams — caolonghua · 2026-10-29
- AI agents double-charged in 70 of 600 trials when payment tools time out — Soggy-Somewhere-8985 · 2026-10-29
- Herdr UI adds native agent chats, phone remote control and scheduled automations — solyarisoftware · 2026-10-29