Running Qwen Flash Next at 262k context on 96GB Strix Halo, seeking speedups
Forward_Jackfruit813 · reddit · 2026-09-05
The author runs Qwen Flash Next locally on a 96GB Strix Halo at IQ4XS with a 262k context limit, offloading NGRAMs to SSD. They're satisfied with the intelligence and now want more speed: 50PP/14Decode near full context. The post shares the full llama.cpp launch configuration (draft-mtp speculative decoding, f16 KV cache, flash-attn, unified KV) and asks whether an m.2-to-OCuLink V620 32GB eGPU would help on the Vulkan build, or if a 27B model might be a faster alternative.
More from Infra
- PostgreSQL 19 slips to November as LLM-assisted code review floods committers with edge cases — sanjaykalra · 2026-10-21
- Decision Models' routing classifier hits AntSeed, cutting inference cost to 24% of always-Opus — airesearch12 · 2026-10-21
- Eon cuts engineer onboarding from 2 days to 2 minutes with an MCP Gateway on Bedrock AgentCore — AWS ML Blog · 2026-10-20
- a16z's Ben Horowitz says AI buildout isn't the 90s bubble — an investor pushes back — henloitsjoyce · 2026-10-20
- Micron and Nvidia drive ~36% of S&P 500's Q3 earnings growth — Beth_Kindig · 2026-10-20
- NVIDIA lands in Europe: Jensen Huang and German chancellor discuss industrial AI at GTC Berlin — nvidia · 2026-10-20