Reddit speculates DeepSeek-style KVCache compression could let 32GB GPUs run 54B-class models
pmttyji · reddit · 2026-09-14
A Redditor sketches a wishlist: if upcoming models adopt DeepSeek-V4.1-Flash-style KVCache + Engram optimizations, local deployment economics change drastically.
- Flash reportedly stores 1M-token KV cache in 1GB; Engram is rumored at 1/3–1/2 of model size (10–15GB for a 30B model, RAM-resident).
- Back-of-envelope table: a Qwen 27B-Q8 with current F16 256K KV cache needs 47GB total; compressing the cache to 1GB cuts it to 32GB (20GB at Q4).
- That implies 32GB VRAM could run Q4 54B-class models, and the author hopes 24GB cards suffice within a year.
Caveat: the "future models" rows are fictional placeholders—this is speculation on local-serving memory math, not an official roadmap.
More from Infra
- Claymation body horror on 8GB: three wrong settings, not the GPU, made local renders slow — roro9- · 2026-10-30
- durable-object-migrator: zero-downtime exports for Cloudflare Durable Objects — samgoodwin89 · 2026-10-30
- An unnamed firm already runs a fairly scaled dedicated vehicle for compute lending, says analyst — matt_slotnick · 2026-10-30
- GPU break-even throughput vs open-weight token prices up 25-83% since May — bookwormengr · 2026-10-30
- Sam Altman says the chip supply chain's invisible hand is failing — and DRAM is why — SnoozeDoggyDog · 2026-10-30
- MoEShift runs a 125B MoE on a 24GB Mac: 7.5GB peak memory at ~10 tok/s — dave-tro · 2026-10-30