Portable Datacenter Rig: Running Qwen3.8-27B with 200K+ Context Locally
Special-Wolverine · reddit · 2026-08-26
A user shared a portable local computing setup for running Qwen3.8-27B-BF16 with over 200K context length, fitting all gear into a lunchbox. The rig features a Ryzen 7745HX CPU and a 96GB VRAM RTX Pro 6000 Blackwell GPU. Benchmarks show a prefill speed of 1715 tok/sec (processing 175K tokens in 102s) and a generation speed of 45 tok/sec, with temperatures staying under 80°C. The author notes that BF16 precision significantly outperforms quantized versions for sensitive tasks like legal document OCR, exceeding the performance of Gemini Pro and ChatGPT 5.6 Sol.
More from Infra
- Lighter crypto system achieves >10x performance boost via optimization — gajesh · 2026-08-26
- Antirez on Mac Studio M5 Ultra: Compute/bandwidth insufficient for K3 tasks — antirez · 2026-08-26
- Evaluating Mac Studio M5 Ultra value by GLM 5.3 concurrency and speed — antirez · 2026-08-26
- EVE Online begins migration: upgrading 2.4M lines of code to Python 3 — Simon Willison · 2026-08-26
- IBM Granite 4.2 8b Blooms to 27GB VRAM Despite 5.2GB Download — x8code · 2026-08-26
- Hardcore Player Builds Underground Data Center Just to Heat Outdoor Jacuzzi — curious_vii · 2026-08-26