Full recipe: running Qwen3.8 27B on AMD Strix Halo with patched ROCm llama.cpp
ilintar · reddit · 2026-09-05
A developer invited by AMD to help with Lemonade published an optimized setup for running Qwen3.8 27B on Strix Halo. Key points:
- Mainline llama.cpp ROCm support is currently broken pending a unified memory PR; he maintains a strix-halo branch merging ring-buffer fixes and a TOP-K optimization.
- A ROCm bug makes graph updates terribly slow; his fix PR is pending, so a custom-built .so is required for now.
- Inspired by Redline, he modified the ROCm HIP library for low-level PM4 dispatch, gaining up to 20% decode for dispatch-bound models, though Qwen3.8 27B on Strix is bandwidth-bound.
- ROCmFP4 is a dead end (no native FP4 on Strix Halo); he instead quantized large tensors to IQ4XS, winning on both quality and kernel performance.
- Base decode caps at 15 t/s; the real speedup comes from IQ4XS-quantized DFlash2 speculative decoding with near-identical acceptance rates.
Everything is packaged for easy installation.
More from Infra
- Dev boosts GLM 5.2 TPS on a B300 and swaps it into Claude Code in place of Anthropic models — abhijithneil · 2026-09-05
- What forces LLM teams to optimize inference when going from MVP to production? — Ok_Philosophy_4031 · 2026-09-05
- Leaker claims GPT-6 Astra beats Claude Fable 5.1 at metaprompting — whoiskatrin · 2026-09-05
- Local Qwen 27B vibecodes a playable Godot dungeon game in just 4 prompts — jacek2023 · 2026-09-05
- Open-sourcing nopasswd-sudo: time-boxed passwordless sudo for agents, built by a local 124B model — max_paperclips · 2026-09-05
- Experiential Labs: open-source gateway unifying 1,000+ AI models with one key and zero markup — Shruti_0810 · 2026-09-05