DeepSeek V4 Flash 0731 hits a ctx_other error in llama.cpp speculative decoding
Ambitious_Fold_2874 · reddit · 2026-08-04
A user is trying to get speculative decoding working with DeepSeek V4 Flash 0731 in llama.cpp, but runs into a context-initialization error during model loading.
Setup
- llama.cpp build: b10228 or earlier.
- Draft model: am17an’s DSPark draft.
- Main model: unsloth/DeepSeek-V4-Flash-0731-GGUF:Q8KXL.
- Multiple NVIDIA GPUs are assigned, with the 5060 Ti cards used for inference.
Error observed
- dflash requires ctxother to be set
- The draft model memory measurement fails before speculative decoding can start.
Why it matters
- The thread is effectively about how to wire up DeepSeek’s flash variant with llama.cpp’s speculative decoding path.
- It includes the exact command line and logs, which may help others reproducing the issue or debugging similar multi-GPU setups.
More from Infra
- chapter-tgz adds chapter boundaries to tar.gz for O(1) skipping and parallel reads — charliermarsh · 2026-08-04
- Community-built MiniMax H3 weights target 12GB–24GB GPUs with INT4 and NVFP4 variants — VoidAsuka · 2026-08-04
- NVIDIA says long-context serving speed is set by architecture choices before training — NVIDIAAI · 2026-08-04
- MiniMax H3 full bf16 run hits 95 GB VRAM and finishes a 15s clip in 2 hours — Moarkush · 2026-08-04
- Windows reset restores RTX 3070 throughput for local Qwen3.6-35B inference — campaigner_ · 2026-08-04
- Emmy cuts Gemma 4-12B TTFT on RTX 5090, but steady-state throughput barely changes — NoVibeCoding · 2026-08-04