DeepSeek V4 Flash Hits 1M Context Locally
Shoddy_Bed3240 · reddit · 2026-07-18
A Reddit user shared their configuration and test results running DeepSeek V4 Flash on a 5090 using llama.cpp, highlighting that they pushed the context length to 1 million tokens.
They used the GGUF version provided by Unsloth and posted the full llama-server startup parameters, including:
- Context length of 1,048,576
- K/V cache quantization
- Various CPU/GPU allocation and batch parameters
- Specifying chat template, repetition penalty, and service port
Measured performance is roughly:
- Prefill around 650–700 tok/s
- Decode around 17 tok/s
- Load time of about 32 seconds
The author believes the speed isn't as ideal as the Qwen series yet, but notes there is still room for optimization in llama.cpp.
More from Infra
- Sail equivalence proof and riscv-coq cross-checks uncover real RISC-V spec bugs — xennygrimmato_ · 2026-10-27
- xAI's Colossus 1 hits ~14.6 exaflops FP64, ~5x Top500 leader — cis_female · 2026-10-27
- Lambda runs 19 B200 nodes in a 16-node power budget, boosting throughput 24% — TheZachMueller · 2026-10-27
- Podcast: Micron's HBM roadmap, Meta's AI infra, and NVIDIA's supply chain lock-in — BenBajarin · 2026-10-27
- SGLang MegaKernel + PD Pushes the TCO-Latency Frontier on GB300, SemiAnalysis Benchmarks Show — AccBalanced · 2026-10-27
- Scaling self-hosted Convex: compiling TypeScript Convex functions to WASM — leventov · 2026-10-27