DeepSeek V4 Flash Local Tested
kevin_1994 · reddit · 2026-07-11
A sharing of local running experiences with DeepSeek V4 Flash on consumer hardware.
The author's setup:
- RTX 4090
- 128 GB DDR5
- Intel Core Ultra 7 270k
- Ubuntu + latest llama.cpp
After testing various quantization and parameter combinations, it was successfully run using unsloth's UD-Q2KXL quantization, yielding these speeds:
- Prompt: 132.5 tok/s
- Generation: 10.9 tok/s
Key observations:
- On the Intel Core Ultra 7 270k, binding p-cores significantly boosted speeds, increasing from roughly 6.8 tok/s to 11 tok/s
- --no-mmap is slower
- -ctk q80 / -ctv q80 crashes llama.cpp
- Raising -b or -ub above 4096 for long contexts inflates the CUDA buffer to 90GB+
- When using llama-server, turning -fa off is necessary, or it also blows up the CUDA buffer
In comparison, the author finds it slightly "smarter" but slower than Qwen 3.6 27B Q4KXL; however, because it exhibits less "overthinking", the total time for many tasks is actually similar. The author believes that once issues like flash attention, batch/microbatch, and context quantization are fixed, this model will become much more practical on 4090/3090 GPUs.
More from Infra
- Tinkerers run GLM-5.2 at near-lossless quality on a $15,000 budget — amplifiedamp · 2026-07-21
- Liquid AI expands a pretrained tokenizer from 65K to 128K without retraining from scratch — JosephJacks_ · 2026-07-21
- AI accelerators now account for 15–20% of active North American data-center power — BenBajarin · 2026-07-21
- An architect’s guide to governing AI in the cloud — bibryam · 2026-07-21
- Gritt raises a new round to automate solar array installation and maintenance — rebeccakaden · 2026-07-21
- Refactoring 150k LOC Takes 96 Hours: Is Compute the Bottleneck for AI Coding? — robleclerc · 2026-07-21