DeepSeek-V4.1-Flash on 4x 3090: 12→25 tok/s Prose, Full Tuning Notes and Gotchas

C0smo777 · reddit · 2026-10-20

• Author benchmarks Unsloth's UD-Q4KXL quant of DeepSeek-V4.1-Flash-1017 on EPYC 9575F + 768GB DDR5 + 4x RTX 3090, improving prose from 12.6 to 24.6 tok/s and code to 39.3 tok/s.
• Key tweaks: loading the whole model into RAM boosts 32K-context generation from 7.5 to 17 tok/s; one thread per physical core stabilizes runs; -ub 2048 speeds prefill 35% for free.
• The DSpark drafter roughly doubles code speed (80% acceptance) but barely helps prose (30% acceptance).
• Detailed gotchas: --fit can't size the drafter (manual --fit-target 4096,4096,4096,7168 works), it conflicts with --override-tensor, and the drafter file must be downloaded by exact filename. Full launch command included; requires llama.cpp PR #30817.

Original post →

More from Infra

• Cadence announces sovereign AI cloud for European chip designers — karlfreund · 2026-10-23
• SGLang Deep-Dive Blog Series Breaks Down KV Cache, Posts up to 15.5x Throughput Gains — BanghuaZ · 2026-10-23
• DynaLM: Open-Source CPU-First LLM Inference Engine Seeks Feedback — Unlikely_Safety_7456 · 2026-10-23
• Capital Is Now the AI Buildout Bottleneck, Says Analyst Citing Jensen Huang — JOBhakdi · 2026-10-23
• Cerebras explains how it deploys trillion-parameter models on wafer-scale chips — bookwormengr · 2026-10-23
• IFM open-sources RL360, the agentic RL infra behind K2 Horizon's training — rupspace · 2026-10-23

Infra channel →