$31/month quad-channel EPYC + 128GB runs 3-bit Qwen at 128k context
Ne00n · reddit · 2026-11-21
A GPU-poor user shares a budget CPU inference setup: a French hosting company sold an EPYC 7351P with 128GB DDR4 for $31/month. Thanks to quad-channel memory bandwidth, running the newest Qwen 3.8 Flash Next 3-bit Unsloth quant yields 3.6–3.8 tokens/s — and crucially the speed holds regardless of context length, sustaining 128k context. He's currently having the model code a game with promising early results, and plans to try the 4-bit quant next (5-bit likely won't fit). A practical reference path for budget-constrained long-context local deployment.
More from Infra
- Inkling 2 Mini quantized to 5.4GB/8.9GB, keeping 91-97% performance — teortaxesTex · 2026-11-21
- Open-weight models are less efficient, so their rise may increase chip sales — aronchick · 2026-11-21
- Score centering, rumored OpenAI RL trick, tolerates staleness and may rescue slow chips — teortaxesTex · 2026-11-21
- Ternary Bonsai-2 vs Qwen3.5-2B on A13/A14/M3: 32-55% faster decode, and why thermal logging matters — leo-k7v · 2026-11-21
- Essay 'The Organic Web' weaves 62 citations to dissect data's physical weight — round · 2026-11-21
- ARK projects Starship marginal launch cost falling to $358/kg by 2027 — mitchdeg · 2026-11-21