DeepSeek V4 Flash Hits 1M Context Locally
Shoddy_Bed3240 · reddit · 2026-07-18
A Reddit user shared their configuration and test results running DeepSeek V4 Flash on a 5090 using llama.cpp, highlighting that they pushed the context length to 1 million tokens.
They used the GGUF version provided by Unsloth and posted the full llama-server startup parameters, including:
- Context length of 1,048,576
- K/V cache quantization
- Various CPU/GPU allocation and batch parameters
- Specifying chat template, repetition penalty, and service port
Measured performance is roughly:
- Prefill around 650–700 tok/s
- Decode around 17 tok/s
- Load time of about 32 seconds
The author believes the speed isn't as ideal as the Qwen series yet, but notes there is still room for optimization in llama.cpp.
More from Infra
- LLM Serving Metrics Thread: Why TPOT and Uptime Make or Break User Experience — abhijithneil · 2026-09-11
- PlanetScale launches sharded Postgres: 768 servers acting as one, 1PB scale — dhruv2038 · 2026-09-11
- Can a 7900 XTX 24GB run Qwen locally? Reddit seeks ROCm tok/s benchmarks — thenomadexplorerlife · 2026-09-11
- RTK Terminal Compression Cuts Tokens but Leaves Your AI Coding Bill Unchanged — Bartaseth · 2026-09-11
- SF Compute founder: buying compute is 'an absolutely awful experience' right now — IgorCarron · 2026-09-11
- SmolVM open-sources persistent computer infrastructure for agents that outlive chat sessions — aniketmaurya · 2026-09-11