48% speedup: optimized llama.cpp settings for Qwen3.8-Flash-Next on Ryzen AI Max+ 395
Prudence-0 · reddit · 2026-11-09
A Reddit user shares llama.cpp launch parameters that deliver a 48% performance boost and more consistent tok/s for Qwen3.8-Flash-Next on a Ryzen AI Max+ 395 with 96GB unified memory, at contexts beyond 64k—without resorting to third-party inference engines.
Key settings include --fit on with --flash-attn, KV cache quantized to q80 with --kv-unified, a 131072 context with --ctx-checkpoints 128, dual parallel continuous batching, and ngram speculative decoding (match 24 / max 16 / min 4). Sampling: temp 1.0, top-p 0.95, top-k 20.
The author also recommends the Qwen3.8-Flash-Next-Uncensored-Gufo-Q4Mix quant as a solid quality/performance balance. The attached screenshot shows 67 requests processing 3.34M prompt tokens with 3.28M cached.
More from Infra
- Appwrite launches MCP1: one URL for every MCP server your team uses — eldadfux · 2026-11-09
- Saudi's HUMAIN and National Infrastructure Fund line up up to SAR 2.5B for AI infrastructure — HUMAIN · 2026-11-09
- Local models' broken chat templates silently broke agent clients — here's the fix — viest_me · 2026-11-09
- Day 309/365 of GPU Programming: Optimizing LLaMA2-7B on Edge with SIMD and Multithreading — NandoDF · 2026-11-09
- Waymo explains at Hot Chips why camera-only autonomy plateaus at 99.9% — 4310sy · 2026-11-09
- Running a Home Mac mini as an Always-On Node for Remote AI Tasks — xiaohu · 2026-11-09