Running 285B DeepSeek-V4-Flash-Vision on 12x RTX 3090s at 120+ tok/s
ciprianveg · reddit · 2026-09-09
Developer ciprianveg published a fully reproducible setup for running DeepSeek-V4-Flash-Vision-Exp (285B MoE) on consumer Ampere GPUs:
- FP4 experts + FP8 attention, 157 GB weights, SM86-compatible vLLM build
- 60+ tok/s decode on 10x RTX 3090 (TP2xPP5, 240W cap) with DSpark speculative decoding (k=3); 120+ tok/s on 12 GPUs (TP4xPP3)
- Vision, tool calls and spec decoding all working; 1M context without offload / 4M with RAM offload, 3,500 tok/s long-context prefill
Ships with a pre-built Docker image ghcr.io/ciprianveg/3090-vllm:dsv4-flash-vision-sm86 and a GitHub repo with build guide, start scripts and runtime patches covering DSpark propose-gate (spec+vision fix), scheduler mm×spec row-crossing, grammar-bitmask validation, ViT OOM fix, FlashInfer workspace-lane keying and more.
More from Infra
- Sail equivalence proof and riscv-coq cross-checks uncover real RISC-V spec bugs — xennygrimmato_ · 2026-10-27
- xAI's Colossus 1 hits ~14.6 exaflops FP64, ~5x Top500 leader — cis_female · 2026-10-27
- Lambda runs 19 B200 nodes in a 16-node power budget, boosting throughput 24% — TheZachMueller · 2026-10-27
- Podcast: Micron's HBM roadmap, Meta's AI infra, and NVIDIA's supply chain lock-in — BenBajarin · 2026-10-27
- SGLang MegaKernel + PD Pushes the TCO-Latency Frontier on GB300, SemiAnalysis Benchmarks Show — AccBalanced · 2026-10-27
- Scaling self-hosted Convex: compiling TypeScript Convex functions to WASM — leventov · 2026-10-27