$31/month quad-channel EPYC + 128GB runs 3-bit Qwen at 128k context

Ne00n · reddit · 2026-11-21

A GPU-poor user shares a budget CPU inference setup: a French hosting company sold an EPYC 7351P with 128GB DDR4 for $31/month. Thanks to quad-channel memory bandwidth, running the newest Qwen 3.8 Flash Next 3-bit Unsloth quant yields 3.6–3.8 tokens/s — and crucially the speed holds regardless of context length, sustaining 128k context. He's currently having the model code a game with promising early results, and plans to try the 4-bit quant next (5-bit likely won't fit). A practical reference path for budget-constrained long-context local deployment.

Original post →

More from Infra

Infra channel →