DeepSeek v4 Flash on 8xH100: vLLM Tuning Bottlenecks
SlipperyCorruptor · reddit · 2026-08-08
A developer encountered performance bottlenecks while deploying the latest DeepSeek v4 Flash model using vLLM on an 8xH100 node. The poster noted that tweaking batching parameters led to a trade-off between prefill and decode phases.
They suspect the core issue lies in scheduling and expert routing, asking the community for pointers on how to extract maximum inference performance from the model on an 8xH100 cluster.
Related event: DeepSeek V4 Flash Local Deployment Faces Hardware and Tuning Hurdles(2 posts)→
More from Infra
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16
- Optuna creator Akiba: when 2 hyperparameters cost $20M, what's left for BO/HPO — Stefania_druga · 2026-10-16
- Amazon Proposes Per-Request Validated Evidence Caching for Production RAG — _reachsumit · 2026-10-16
- LinkedIn's TransX report: sequence transduction for recsys, +6.02% CTR in A/B tests — _reachsumit · 2026-10-16
- llama.cpp PR boosts CPU inference speed with merged AVX2 tiled accumulators — jacek2023 · 2026-10-16
- Zero's next-gen IVM engine runs in a single DO, live-querying a 100GiB R2 database — threepointone · 2026-10-16