Rust HIP inference engine targets dual R9700 RDNA4 with hot-expert routing
Public_Umpire_1099 · reddit · 2026-08-04
A Reddit user is building a custom Rust + HIP inference engine for DSV4F on a dual-R9700 RDNA4 setup and is looking for a robust multi-GPU arrangement.
Key details:
- Current setup: 2× R9700, Ryzen 5 9600X, 128 GB DDR5
- Main bottleneck: the second card is only on PCIe 4 x4
- Planned approach: keep hot experts on the faster GPU, use system RAM for cold/missed experts, then swap them back into GPU memory during cooldown periods
- Early benchmark: about 80 tok/s best case with DFlash, hoping for at least 60 tok/s in the target setup
This is a practical thread about inference-engine design, bandwidth constraints, and expert placement for a very specific RDNA4 multi-GPU stack.
More from Infra
- Multi-agent systems may overwhelm laptop RAM and CPU, poster warns — zephyr_z9 · 2026-08-04
- Doota Mail self-hosting is now fork, secrets, and CI rerun — samgoodwin89 · 2026-08-04
- FL2VA 20B is the default local pick for video generation on a normal PC — cocktailpeanut · 2026-08-04
- Startup routes inference across desktop devices and shards models across two PCs — punkyrockypocky · 2026-08-04
- Wan2GP launcher targets AMD GPUs for one-click local generation — cocktailpeanut · 2026-08-04
- Pinokio adds an AMD-only Wan2GP launcher for 6GB+ VRAM PCs — cocktailpeanut · 2026-08-04