DeepSeek-V4.1-Flash hits ~30 tok/s on a 12GB GPU with expert-skipping tricks

midokino · reddit · 2026-11-21

Reddit user midokino details running DeepSeek-V4.1-Flash (ISTA-DASLab IQ2XS 2.25 bpw, 266GB) on a gaming PC (RTX 5070 12GB, Ryzen 7 9700X, 64GB RAM), open-sourced as a Strata fork. From 5 tok/s out of the box, tuning reached 32.5 tok/s on code and 28 tok/s on mixed tasks, with 519 tok/s prompt reads on 30K tokens. Quality: GPQA Diamond 89.9% (quant card: 90.40, original: 90.91), HumanEval+/EvalPlus 94.5/90.2%. Key optimizations: skipping low-weight experts that would come from SSD (main speedup, source of KLD 0.183 vs 0.057 conservative preset with equal scores); capping the DSpark draft model's VRAM share (code +29%, thinking +22%); closing buffered file handles after load (+7-17% prompt reads); Expert Deferral from KTransformers SOSP '25 (+11%). He's asking for someone with 500GB capacity to measure KLD against the original model.

Original post →

More from Infra

Infra channel →