Running llama.cpp on NVIDIA + AMD GPUs: build separate CUDA and HIP binaries

espece-de-bon · reddit · 2026-10-13

A Reddit thread on running llama.cpp with both NVIDIA (CUDA) and AMD (HIP/ROCm) GPUs in one machine:

• HIP can't be added to a CUDA build at runtime; it needs a separate compile with -DGGMLHIP=ON via the ROCm toolchain. Two builds coexist fine in separate build directories.
• Recommended: keep the CUDA build for the NVIDIA card, add a sibling HIP build for the Radeon, and run two server instances on different ports — best stability and per-card performance.
• Splitting one model across both (e.g., 16GB:32GB via build-both, -sm layer -ts 16,32) is experimental and may not deliver expected memory savings due to a duplication bug.

Concrete cmake commands included.

Original post →

More from Infra

• Starlink Mini saves five at sea after sinking boat's radio fails — elonmusk · 2026-10-13
• NVIDIA maps France's AI ecosystem: 50-satellite constellation, Bull to build Vera Rubin servers — nvidia · 2026-10-13
• NVIDIA maps France's agentic computing push: satellites, sovereign models and quantum links — NVIDIA Blog · 2026-10-13
• LangChain built SmithDB, a new database just for agent traces in LangSmith — hwchase17 · 2026-10-13
• IMF Forecast: AI Adds No Measurable Productivity Yet, Bust Would Cost Twice the Boom's Gain — rohanpaul_ai · 2026-10-13
• NVIDIA ships a NURBS B-rep kernel on USD, but devs slam its nondeterministic geometry — rms80 · 2026-10-13

Infra channel →