Running 748B DeepSeek-V4.1-Flash on a €2k used PC at 7.2 tok/s
Vladowski · reddit · 2026-10-22
A user got the 748B DeepSeek-V4.1-Flash-1017 running on a used sub-€2,000 PC (RTX 3090 24GB, 128GB DDR4, i9-9900X) at 7.2 tok/s with the smallest 1-bit quant (Unsloth UD-IQ1S, 233 GB on disk).
The key trick: the 110 GB Engram table isn't loaded up front — it stays on NVMe and only needed rows are read on demand. Built from open llama.cpp PR #30817 with CUDA 12.8.
The author admits the output contains a physics error and has limited practical use (Qwen models work better on his setup), but it's a fun demonstration of fitting a massive MoE on modest hardware.
More from Infra
- GitHub moves 99.99999% of repos to Azure; per-reference quorums fix 25ms latency wall — rseroter · 2026-10-22
- Flint model weights hit HuggingFace, runs on DGX Spark, RTX GPUs and M4 Pro — testingcatalog · 2026-10-22
- ABB ships RobotStudio HyperReality on NVIDIA Omniverse; Absolute Accuracy calibration is key — lukas_m_ziegler · 2026-10-22
- Flappy Airplanes raises $1B Series A co-led by Index and Lux for new GPU primitives — gabriel1 · 2026-10-22
- Researcher: the final megakernel isn't a kernel but a transformer in an endless loop — DimitrisPapail · 2026-10-22
- SPCX earnings call highlights 'computer per watt' metric as AI value shifts down the stack — mobileraj · 2026-10-22