DeepSeek V4 Flash fits on a 128GB Ryzen AI MAX+ 395 and hits 32 tok/s
sandropuppo · reddit · 2026-07-28
A Reddit post shows how DeepSeek V4 Flash was fit onto a single AMD Ryzen AI MAX+ 395 system with 128 GB of unified memory, reaching up to 32 tok/s with speculative decoding.
Key details:
- The model was compressed with the ROCmFPX family of block formats, using a mixed-precision recipe that brought the target to about 102.3 GB, or roughly 2.88 bits per parameter.
- With no speculative draft, the DeepSeek-specific HIP decode path reached 25.31 tok/s autoregressively.
- Adding a small DSpark draft and a q=4 verification cap pushed throughput to 32.0 tok/s.
- Sparse prefill reached roughly 245 tok/s in the published run, with separate validation cases ranging from 246.8 to 255.9 tok/s around the 8K context window.
The post also notes that sparse prefill is opt-in because its reduction order is not byte-identical to exact tokenwise prefill, even though small smoke tests passed.
More from Infra
- Meta releases Muse Spark Open, trained with vLLM-powered RL inference from day one — colinmcnamara · 2026-10-16
- Meta's open Muse Spark needs 493GB of memory — far beyond any consumer laptop — TejasKumar_ · 2026-10-16
- CoreWeave plunges 20% as credit markets price >60% default odds; is this dot-com all over again? — kimmonismus · 2026-10-16
- MiniCPM-V 4.7 35B-A3B hits 17+ tokens/s on iPad with ~3GB memory — techNmak · 2026-10-16
- China rolls out 'token factories' and a 2026-28 token economy action plan — nordicinst · 2026-10-16
- Vercel Sandbox Drives go GA: attach TBs of data in sub-second, fork instantly — cramforce · 2026-10-16