Two mining rigs, one LLM: GLM-5.3-Flash hits 42 tok/s split across 8x RTX 3060 over 1GbE

alexlucaci · reddit · 2026-11-05

A Reddit user's Project Maya patch splits GLM-5.3-Flash (Maya-S24 quant) layer-wise across two X79 boxes with 4x RTX 3060 each: box A runs layers 0-23 plus tokenizer/API, box B runs layers 24-44 plus MTP head/sampler, with only 25KB of hidden state per token crossing a plain 1GbE link (0.3ms).

Results: 38-42 tok/s decode, 1,100 tok/s prompt reading at 30K context, 6-7s per agent turn; scores the same 48/48 on a coding-agent benchmark as a single 8-GPU box, with network overhead only 1-2%.

Key lesson: RAM, not the network, was the bottleneck — at 16GB per box expert weights spilled to SATA SSD (31s turns); bumping to 20+32GB eliminated SSD reads entirely. The 1,900-line patch is open source with IP allowlist and HMAC-SHA256 auth. On a one-shot hard suite it scored 6/10, with 3 misses caused by the 32K token budget, not wrong code.

Original post →

Two mining rigs, one LLM: GLM-5.3-Flash hits 42 tok/s split across 8x RTX 3060 over 1GbE

More from coding & agent