Dual RTX PRO 6000 Benchmarks: DeepSeek-V4-Flash vs Qwen3.8-27B Local Serving
dee_hw · x · 2026-10-22
autonomouslabs shares real-world numbers from their Autonomous Computer 2 (2× RTX PRO 6000 Blackwell, 96GB each / 192GB total, vLLM 0.26.1) serving two open models to 16 concurrent users:
- DeepSeek-V4-Flash (both cards, tensor parallel 2): 753 tok/s total output, 47 tok/s per user, 1.5s to first token, 161 tok/s single-user, 280K token KV pool
- Qwen3.8-27B (one card, FP8): 893 tok/s total output, 61 tok/s per user, 0.42s to first token, 80 tok/s single-user, 1.52M token KV pool
The takeaway: excitement over new model releases fades once you check the weights — the best model is the one your hardware can actually serve. Measured locally with separate runs and different workloads.
More from Infra
- One Cloudflare Durable Object at $9/month replaces Supabase's ~$33K real-time sync plan — threepointone · 2026-10-22
- SF Systems Meetup: Agents-Only Version Control and Postgres Rewritten in Rust — ShadajL · 2026-10-22
- Cloudflare saw thousands of php2shell exploit attempts before disclosure, millions now — ivan_bezdomny · 2026-10-22
- Jensen Huang: NVIDIA Is "Literally the Only One" You Can Trust With a $100B Blind Chip Order — jfischoff · 2026-10-22
- 2-bit draft in VRAM + residual in RAM: speculative decoding when the model doesn't fit — Happy_Guitar3521 · 2026-10-22
- Korean neocloud Elice now up 68% from IPO-day low — the panic headlines have gone quiet — firstadopter · 2026-10-22