4x DGX Spark cluster recipe runs GLM-4.6-Flash with 1M context, 16 concurrent, fully local
AIFlow_ML · x · 2026-10-25
Developer tenhkspark published cluster-recipe (GitHub, MIT): a fully local inference setup on 4x DGX Spark (GB10, 128GB unified memory each) running GLM-4.6-Flash with 1M context and 16 concurrent sessions.
- The official install path couldn't reach that context/concurrency, so the author swapped the loader; the modified code was audited for safety.
- The cluster has four seats: two large reasoning seats, one fast worker seat, one document/retrieval seat — all NVIDIA open models, every component smoke-tested on real hardware.
- Includes build instructions for Nemotron Parse (a lightweight document-image model not officially supported on GB10), with gotchas and measured benchmarks.
- Ships perf-test and burst-test scripts; READMEs in English, Chinese, Japanese and Korean.
More from coding & agent
- Open-source decision model Laya hits 32K phones in one month at ~33ms per call — HowDevelop · 2026-10-25
- Agents Mirror Your Org Chart Just Like Microservices Did, Engineer Argues — arpit_bhayani · 2026-10-25
- AI-built bbtree claims fastest concurrent btree; researcher calls for ambitious academic systems — tianyin_xu · 2026-10-25
- Froxel: open-source voxel editor with built-in MCP server lets AI agents model and render — roman_va · 2026-10-25
- CCC dashboard treats 'waiting on human' as a separate state from agent busy — amirfish · 2026-10-25
- Octopus OS: a 45k-line Rust kernel where agents rewrite their own harness mid-task — _AustinCalvert_ · 2026-10-25