Qwen 3.5 9B + DFlash hits 75 tok/s on R9700: Full Setup Guide
karmakaze1 · reddit · 2026-08-20
A Reddit user shared a detailed guide to achieving 75 tokens/sec inference speed with the Qwen 3.5 9B model on an R9 7900 GPU. The setup combines a fine-tuned model (DavidAU/Qwen3.5-9B-The-Defiant-Fable...) with DFlash draft model acceleration, utilizing Q80 quantization and 64K context.
Key steps include:
- Tokenizer Prep: Downloading official Qwen BPE files to the DFlash source directory.
- Conversion: Converting the draft model to GGUF format using converthftogguf.py.
- Config: Updating the INI file to use model-draft and spec-draft-n-max for Speculative Decoding.
- Execution: Providing a CLI command to launch the server.
This approach leverages a draft model to significantly boost generation throughput.
More from coding & agent
- Grok Bot Ran a 12-Hour Data Analysis Nonstop, Handling API Rate Limits Itself — Daniel_Farinax · 2026-10-16
- Apple code signing drags for hours; official advice: just wait a week — pjausovec · 2026-10-16
- Hermes: a 4B model with 8K context got worse with more test-time compute, pointing to contextual reasoning — mangahomanga · 2026-10-16
- Flux 3 Klein's Default Templates Are Broken Without Its Prompt Upscaler — StrictMove5839 · 2026-10-16
- 21-person team ships hundreds of PRs weekly with AI writing 97% of the code — rseroter · 2026-10-16
- ClosedMCP: Set Up MCP Servers Once, Share Them Across Your Whole Team — Ok_Sprinkles1301 · 2026-10-16