Startup Builds $18,000 Local AI Server With 128GB VRAM, Runs 16 Concurrent Sessions
bendee983 · x · 2026-10-14
A 10-person startup built an $18,000 local AI server — one Threadripper box with 4× Radeon AI PRO R9700 (128GB total VRAM) running Qwen3.8 and DeepSeek V4 Flash, GPUs undervolted below 210W. It handles 16 concurrent sessions at 538.5 tok/s aggregate (48K context each) or 365 tok/s at 128K context.
While some quip the money equals 8 years of Claude subscriptions for worse models, @bendee983 argues these experimenters will win long-term: local models keep improving, AI lab subsidies will end (full token pricing is coming), and startups should hedge accordingly.
More from Infra
- llama.cpp ships inside Google's Android AICore powering Gemini Nano — ggerganov · 2026-10-14
- FLAT brings open-source linear attention training kernels to TPUs, up to 3x faster — SonglinYang4 · 2026-10-14
- Artist built a 300-clip immersive exhibition with MiniMax H3, benchmarks GPU render times — komarco · 2026-10-14
- Nvidia reportedly to announce $28B AI investment in Europe this week — rchardkovacs · 2026-10-14
- RISC-V crypto accelerator SoC gets full place-and-route done with ~$2,000 of LLM tokens — IgorCarron · 2026-10-14
- SK Hynix has struggled with HBM4 back-end issues for over a year, Samsung tipped to lead HBM market next year — zephyr_z9 · 2026-10-14