Running 4 SSD-hybrid inference rigs with RAID10 mirroring across cards
TheZachMueller · x · 2026-10-12
TheZachMueller quotes a Japanese post about "the dawn of SSD-hybrid inference," saying he was "too early again": he has 4 rigs running, with RAID10 mirroring across card pairs to serve SSD-hybrid inference.
It reflects community exploration of using large SSDs to hold weights exceeding VRAM and cut inference costs — hardware ahead of the software ecosystem, but an interesting local-infra experiment.
More from Infra
- 10 open-source GitHub repos to cut your AI agent's token bill — victor_explore · 2026-10-12
- ASML drops bookings, TSMC's CoWoS allocation, record chip packaging price hikes — tengyanAI · 2026-10-12
- saranormous: Inference Is a Scale Game Now, Incumbents Rarely Churn Customers — saranormous · 2026-10-12
- Free AI chat Auro adds cross-device sync, web search and 30+ voices — TheMoonMidas · 2026-10-12
- Yandex's Vladimir data center halted after UAV attack, cloud in emergency mode — AccBalanced · 2026-10-12
- Decapybara: open-source C++20/CUDA rebuild of MegaCapybara inference server — giveen · 2026-10-12