Buying CMP mining GPUs for local LLMs? Bandwidth is the real bottleneck
StellarWox · reddit · 2026-09-01
A Redditor considering $1000 of NVIDIA CMP 100-210 mining cards for local LLM inference asks whether the cards' notoriously slow memory bandwidth matters if the model fully fits in VRAM, and whether multi-GPU layer-splitting only needs to pass activations between cards rather than weights.
More from Infra
- Buying GPUs for learning quantization and inference, not just to replace APIs — max_paperclips · 2026-09-01
- Opinion: 'Data centers' should be rebranded as 'compute centers' to reflect AI reality — gandamu_ml · 2026-09-01
- RTX 5090-Optimized Qwen3.8 Hits 262K Downloads in 17 Days on Hugging Face — const_reborn · 2026-09-01
- VMware Explore: Broadcom Unveils Private AI Cloud to Tackle Production Friction — DavidLinthicum · 2026-09-01
- Musk debunks 'orbital AI impossible' claims, critics lack basic physics knowledge — elonmusk · 2026-09-01
- Podcast: Does using AI chatbots actually raise your carbon footprint? Probably not — AndyMasley · 2026-09-01