Troubleshooting Multi-GPU Support for Deepseek V3 in llama.cpp
erazortt · reddit · 2026-08-22
A user is struggling to optimize llama.cpp for Deepseek V3 (120GB quantized) across two GPUs (Blackwell 5000 48GB + RTX 3090 24GB). Attempts to use --split-mode layer or manual tensor offloading resulted in allocation failures or slower speeds compared to running on a single GPU with CPU offload. The post seeks advice on correct multi-GPU configuration.
More from coding & agent
- Before building high-stakes AI systems: tell the LLM not to build anything — aryanXmahajan · 2026-08-22
- AI Refactoring Risks: Accidentally Deleting Valuable Technical Debt — sebpaquet · 2026-08-22
- MiniMax M3 ran overnight and found a TypeScript compiler crash—fuzzing first, then static analysis — DanielLockyer · 2026-08-22
- 10 Claude Code Steering Mechanisms That Stop Agents From Ignoring Instructions — bibryam · 2026-08-22
- DeepMind's Aletheia Agent Solves Math Conjectures via Self-Verification — thisguyknowsai · 2026-08-22
- Designing the Backend for Agent Systems: API, Queues, and Cost Control — kmeanskaran · 2026-08-22