A $2.5k local rig runs Qwen3.8 27B at 150 tokens/s, benchmarks show
casper_hansen_ · x · 2026-11-05
casperhansen shares real-world decode speeds from a $2.5k local inference build: Qwen3.8 27B at 150 tok/s (3.5bpw), DeepSeek V4.1 Flash 1017 at 15 tok/s (2.25bpw), and GLM 5.3 Flash at 10 tok/s (3.05bpw). Benchmarks compare nvme vs nvme-exact modes across 1k/8k/32k contexts, with an upgrade targeting 25 tok/s for the slower models.
Related event: $2,500 Local Rig Runs 27B Model at 150 tokens/s(2 posts)→
More from Infra
- Debian patches apt flaw letting MITM attackers swap packages post-verification for root — jedisct1 · 2026-11-05
- Neocloud and Powered Shell Stocks Dip as Uncertainty Over Powered Land Deals Steps Up — toptickcrypto · 2026-11-05
- Unsloth Dynamic v3.1 GGUFs boost Qwen3.8-Flash-Next accuracy 10-15% over GSQ-RCO — danielhanchen · 2026-11-05
- Arena data: Gemini 4 Carbon scales cheapest on hard tasks, rivals cost 3-4x more — arena · 2026-11-05
- Apple warns holiday-quarter gross margin will slide to 45% as memory chip prices surge — rohanpaul_ai · 2026-11-05
- AI infra is token-centric, but pixels, video and 3D deserve first-class status — prateekj · 2026-11-05