Dev Plans Free Community Access to Local Qwen 3.8 27B at 56 Tokens/sec
No_Run8812 · reddit · 2026-08-18
A developer plans to offer free community access to a locally hosted Qwen 3.8 27B (8-bit quant). Running on a single RTX PRO 6000 96GB via vLLM, it achieves 56 tk/s decode with 3 concurrent users at 262K context. The author is soliciting feedback on context length trade-offs and access methods (API vs WebUI).
More from Infra
- First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16