Qwen3.8 quant testing on 5090/Strix Halo: Gufo engine fixes tool-calling failures
awitod · reddit · 2026-10-20
The author benchmarked many Qwen3.8-Flash-Next quants on a 5090+4090 box (152GB total RAM) and a 128GB Strix Halo. Best results came from ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (Q3S); larger unsloth quants had frequent tool-calling failures. Switching to Gufo, an open-source Strix Halo inference engine (810 GitHub stars), eliminated the issues on Strata — but it's Strix-only. Strata's QFN-specific LlamaCpp tweaks still hit a class of bad outputs. The author is seeking equivalent options for CUDA. Useful reference for local deployment.
More from Infra
- PyTorchCon North America 2026 Opens in San Jose With Seven Tracks — PyTorch · 2026-10-20
- Tried to buy an RTX 5090 under $10k this week — couldn't find a single one — CedricMakes · 2026-10-20
- Amazon says its AI chips will support popular coding languages, challenging Nvidia — pstAsiatech · 2026-10-20
- Q-CTRL: over 99.5% of qubits sit idle in one fault-tolerant Shor's algorithm blueprint — MJBiercuk · 2026-10-20
- Token-maxxing is dead: OpenRouter's token usage plateaus — pranshuchittora · 2026-10-20
- Nscale and Anthropic reportedly in talks for another $45 billion compute deal — TechCrunch AI · 2026-10-20