Running DeepSeek V4 Flash 4-bit on Low-End Hardware: A Hardcore Experiment
Similar_Can_3143 · reddit · 2026-08-23
The author shares experiments running DeepSeek V4 Flash 4-bit quantization on constrained hardware (2x 3090+3060, 128GB RAM). By modifying llama.cpp logic to eliminate redundant caching between RAM and VRAM, and adopting a two-stage strategy (lower quant model for prompt processing + original model for generation), they achieved 20+ tgs generation speed and nearly 200 t/s prompt processing speed.
More from coding & agent
- "Vibecoding" is as misleading a label as "AI" and "SI," dev complains — eschadiol · 2026-10-16
- Doc2Harness: New benchmark finds best coding agent passes only 53% across 329 cases — _akhaliq · 2026-10-16
- Perplexity open-sources SPACE, the Firecracker microVM sandbox behind Perplexity Computer — perplexity_ai · 2026-10-16
- Blackstone goes all-in on Factory for autonomous software development — matanSF · 2026-10-16
- Longtime Windsurf user cancels $10/month legacy plan, citing AI subscription fatigue — tristanbob · 2026-10-16
- 'Dropping a 3,000-line PR twenty minutes after ticket assignment' — AI coding meme goes viral — facontidavide · 2026-10-16