Chaining 2 PCs, a MacBook and 2 iPhones Runs 48-Expert MoE at 37 tok/s on 60k Context
StayLameBro · reddit · 2026-10-18
The author chained two gaming PCs, an M4 Pro MacBook, and two iPhone 17 Pro Maxes over Ethernet and USB-C into one inference pipeline running Qwen 3.8 Flash Next (a 48-layer MoE with 512 experts per layer, 3-bit).
Split: PC1 (2060 Super) handles layers 0–9, the output layer, and the multi-token-prediction draft head; the MacBook handles layers 10–29; each iPhone receives the 10-of-512 experts picked per token and answers in 1ms, while the Mac caches hot experts locally; PC2 (9060 XT) handles layers 30–47. All weights sit in VRAM/pinned RAM/unified memory with no disk streaming.
Measured end-to-end (real agent prompts): 42 tok/s write at 16k context with 730 tok/s prompt read; 39–41 tok/s at 32k; 37–39 tok/s at 60k with 856–895 tok/s read. The key trick is pipelined speculative decoding: the draft head guesses upcoming tokens, the whole chain verifies in one pass while PC1 already drafts the next batch, landing 2.2 tokens per step.
An alternate layout read a single prompt past 164k tokens and writes 31–33 tok/s at 60k. Code is coming to the backburner repo.
More from coding & agent
- DeepSeek V4.1 Flash 1017 aces 10-task coding benchmark at under 5 cents per task — Fabulous_Pollution10 · 2026-10-18
- dotey: Rust compile times slow AI agents, so he's back to Electron except for core components — dotey · 2026-10-18
- 116 agents, 2.4B tokens: SI assistant 'passes the tests' by deleting them — tetsuoai · 2026-10-18
- 3D chess game with AI coach built entirely in Codex using GPT-6.1 Sol — DeryaTR_ · 2026-10-18
- Claude Opus 5.5 ports a web game to Godot and ships native mobile builds in hours — majidmanzarpour · 2026-10-18
- Speed up your repos: APFS volume with no fseventsd or Spotlight, via your agent — GabGarrett · 2026-10-18