How One Company Serves 40 Staff Local AI on 4x RTX 6000 Pro With vLLM and n8n
q-admin007 · reddit · 2026-10-13
A redditor details running local AI for 40 employees (10 developers) on a 2U server with 4x RTX PRO 6000 Blackwell Max-Q: DeepSeek V4 Flash via vLLM (TP4, 1M context, 0.55 GPU utilization) as main model, Qwen 3.8 27B Q8 fallback, K2-Horizon 7B mini model; n8n automation with Whisper+Vexa AI Teams transcription bots; OpenWebUI for staff with agentic features disabled; BiFrost OpenAI-compatible routing for devs; ComfyUI for marketing media; BGE-M3 embeddings and reranker via llama.cpp.
More from coding & agent
- How Every cut its AI agent's token costs: caching, less reading, Opus 5.5 — every · 2026-10-13
- 81% of engineering leaders hit AI-related production incidents in six months — sanjaykalra · 2026-10-13
- mitsuhiko on mixing models: GPT 5.6 works, Astra didn't make the cut — mitsuhiko · 2026-10-13
- Full-chain Chrome exploit with all bugs found by agents in six days — moyix · 2026-10-13
- RCM agent build: 59/60 accurate, $250/month, 6 weeks on safety not tokens — alex_verem · 2026-10-13
- Cloudflare challenges devs to build the next GitHub for AI agents — samgoodwin89 · 2026-10-13