Can llama.cpp RPC span a Mac mini, Mac Studio and dual RTX 3090s to run a 120B model?

ruffus_or · reddit · 2026-11-22

A user is planning a three-machine distributed AI cluster to run a 120B+ parameter model: a Mac mini M4 Pro (64GB unified memory), a Mac Studio M4 Max (36GB unified memory) they're considering buying, and a Windows PC with 2× RTX 3090 (48GB VRAM) plus 64GB DDR4 — 212GB of physical memory in total, though not a single shared pool.

They intend to combine Metal, CUDA and CPU offload via llama.cpp RPC or similar, with Thunderbolt 5 between the Macs and 10Gb Ethernet to the Windows box, and are asking whether anyone has pulled off something comparable.

Original post →

More from Infra

Infra channel →