Running a 125B MoE across two PCs: Strata and llama.cpp RPC feasibility question
objectivelywrongbro · reddit · 2026-10-31
A developer asks whether Strata can split Qwen3.8-Flash-Next (125B-A6B) across two machines. Their main rig (Ryzen 7 7800X3D, RX 9070 XT 16GB, 32GB DDR5) runs Qwen3.8-27B (IQ3M) on llama.cpp at 65 tok/s via Vulkan/ROCm. They want to know if expert weights can live on a second PC's 32GB DDR4, whether llama.cpp RPC over LAN for spread experts is realistic or too slow, and request real tok/s and time-to-first-token numbers for experts in system RAM.
More from Infra
- Cloudflare Durable Objects No Longer Interrupted by Deploys, Update on Hibernate — threepointone · 2026-10-31
- Google quantum chip solves in 5 minutes a task that would take 10 septillion years — acmoytoy · 2026-10-31
- Claude Code background agents burn ~$4.80/hour just writing status lines, dev calculates — arpit_bhayani · 2026-10-31
- Two weeks of agent logs: INT4 quantization triples Qwen 27B's reasoning loops at long context — Old_Ad_6033 · 2026-10-31
- YOLOv2 Face Detection Hits 30fps on a K210 Chip via Custom C++ Firmware — alexcovo_eth · 2026-10-31
- Playable Environments: Millisecond Cloud Branching as the Next Test Bed for Agents — matei_zaharia · 2026-10-31