DeepSeek-V4-Flash on 4x Radeon AI PRO R9700: the 'refused' 4-card config hits ~79 tok/s
brainchillzZ · reddit · 2026-10-16
A r/LocalLLaMA user verified that the affinity inference engine — which refused to start on more than 2 cards after its author saw repeated-token garbage in a 4-rank simulation — runs cleanly on four real Radeon AI PRO R9700s with full P2P on a Threadripper PRO 9955WX.
Key numbers:
- 2 cards: 36 tok/s decode, only 45% of experts in VRAM, 82% of token time waiting on host streaming from a 51 GB pinned pool
- 4 cards: 79 tok/s greedy, 63 tok/s at temp 1.0, 90 tok/s on a chat answer with thinking on; all 11,008 experts resident, host pool never allocated
Five greedy and sampled runs plus a chat-server test produced no degenerate output, and the engine's cross-rank consistency check passed every time — the bug lives in the 4-on-2 simulation, not 4-rank execution. Gotchas: pass --placement-strategy static on 4 cards (default heat placement aborts), and load takes about a minute, 42 seconds of which is pushing 96 GB of experts onto the cards. The author also corrected his earlier guess: affinity's collective already supports N ranks (tested at 3, 4, 8).
More from Infra
- Phison CEO calls CXMT memory scare a 'ghost story', warns of materials shortage — BenBajarin · 2026-10-16
- Ben Bajarin: The Future of Compute Is Fungible as Hyperscalers Design for Workload Flexibility — BenBajarin · 2026-10-16
- AirLLM runs frontier-size models layer-by-layer: 671B on 12GB VRAM — Aiden_Tech_Ai · 2026-10-16
- NVIDIA Launches cuPotential in CUDA-X to Bring AI Agents to Semiconductor Materials Simulation — tomaszbednarz · 2026-10-16
- Q4-quantized coder model on 8-core CPU builds interactive solar system demo in one shot — IntrepidMindExplorer · 2026-10-16
- TokenSpeed MLA 2 ships for Vera Rubin: up to 3.44× faster Kimi K3 decode — zhyncs42 · 2026-10-16