Qwen3.8 Flash-Next on one RTX 5080: 250 tok/s agent coding, 5x faster than llama.cpp
yehyakar · reddit · 2026-11-06
A detailed Reddit guide runs Qwen3.8 Flash-Next on a single RTX 5080 (16GB) + 64GB RAM, hitting 195.6 tok/s average decode and 5,700 tok/s prefill in a real 151-request agent coding session with up to 202K context.
The stack:
- Basalt engine: 3.5x llama.cpp decode without drafting, 5x with, 7x faster prompt reads on the same model file.
- Unsloth's new UD-IQ3XXS quant: same size as the popular IQ3S but wikitext perplexity drops 3.382→3.247, nearly matching BF16 (3.273), at 1-3.5% speed cost.
- Lamina MTP drafter: draft acceptance 83.1%→85.8%, zero quality impact.
The author also upstreamed Basalt optimizations: KV-cache spill stalls cut from 0.6-2.4s to 75ms, huge-page compaction pins from 1.5-3.7s to 23ms. A setup script automates the whole build (90GB download, SHA-256 verified; needs CUDA 13, 64GB RAM, 180GB disk).
More from coding & agent
- Gemini /teamwork agents caught live: four parallel task logs all running that second — RyanSpunk · 2026-11-13
- Codex CLI 0.168.0 adds model provider picker, side-by-side TUI workspace and animated GIF generation — github-actions[bot] · 2026-11-13
- Free MCP server lets Claude read Altium, Allegro, KiCad and Gerber PCB files directly — woodisruff · 2026-11-13
- a16z Partner Clones Open-Source Game with Fable 5.2 Agent, Which Finds Its Own Horse Animation Asset — FinanceYF5 · 2026-11-13
- OpenAI team shares how it runs and governs fleets of persistent agents internally — romainhuet · 2026-11-13
- Perplexity's Decider scanned 1B tokens across 7,000 agent sessions, found 14 security fixes — denisyarats · 2026-11-13