Routing NVIDIA PAIR to llama.cpp on an AMD ROCm node (2×R9700): full notes
Don_Reuter · reddit · 2026-09-11
A detailed writeup of making NVIDIA's Personal AI Router (PAIR) route to a tuned llama.cpp setup on an AMD ROCm node. PAIR ships GPU telemetry only for NVIDIA nodes, so the author added AMD telemetry via amd-smi (ROCm 7.2.0) and fronted an existing llama-swap server with a custom process-mode engine manifest + adoption. Verified on 2×R9700 (gfx1201) under the Go race detector. Code was vibe-coded by a local Qwen model with local-model review. Gotchas: manifests are embedded at build time (no rebuild = no effect), proxies are per-protocol not per-node, and the Intel xpu-smi changes are untested. A practical guide for heterogeneous local-inference clusters.
More from Infra
- SemiAnalysis: OpenAI reportedly negotiating ~1GW TPU commitment while seeking sites — dylan522p · 2026-10-28
- I tested 11 video upscaling methods for AI video: local, Topaz and cloud compared — lopas99 · 2026-10-28
- Thermo-backed CTO says AI is in its dial-up phase, touts disaggregated compute for an ultrafast 'broadband moment' — beffjezos · 2026-10-28
- DeepSeek-V4.1-Flash Q2 runs at 41 tok/s on one RTX 3090, 90% token match vs API — musicurgy · 2026-10-28
- USearch v3 RC ships after 1,000 days, its libraries now embedded in 550K-star projects — viglovikov · 2026-10-28
- SK Hynix says memory demand will outpace supply through 2030 — Chemical_Hawk_6307 · 2026-10-28