Can PCIe switches help GPU+RAM+NVMe hybrid inference setups?
HankYeomans · x · 2026-10-25
HankYeomans asks whether PCIe switches apply to his hybrid inference recipe: DeepSeek 4.1 flash with GPU+CPU/RAM+NVMe tiering.
Per the original thread, the multi-GPU-per-slot approach assumes no RAM offloading and P2P enabled—conditions a hybrid RAM/NVMe setup may violate.
More from Infra
- Jensen Huang unveils Vera Rubin DSX POD at GTC Berlin, pitches Europe's AI economy — nvidia · 2026-10-25
- TensorSharp ships Day-1 MiniMax-M3.1-Flash support: 9.8× faster 64K prefill than llama.cpp on 8× A40 — fuzhongkai · 2026-10-25
- ANEMLL Forge 0.2.1 runs Qwen3.8-27B at up to 192K context on M6 Mac Mini's Neural Engine — Competitive-Bake4602 · 2026-10-25
- Compute contracts vs reality: Nscale has only 5% of contracted GPUs live, TeraWulf 12% — AccBalanced · 2026-10-25
- Live MCP rewrite from Node to Rust cuts memory usage by roughly 8x — draginol · 2026-10-25
- 305M-Parameter Sparse MoE Runs Computer-Use Agent Fully Offline on a MacBook Air — lmoroney · 2026-10-25