colibrì Tested: Running GLM-5.2 Without a GPU
Lopsided_Dot_4557 · reddit · 2026-07-15
The author conducted full tests using the pure C colibrì engine, attempting to run GLM-5.2 744B MoE on a single machine without a GPU.
Key Concepts
- The 744B MoE only activates a few experts per token.
- The engine keeps the dense components (around 10GB) in memory.
- The remaining expert weights are streamed on-demand from local disk/NVMe.
- The full int4 model is about 370GB, but it doesn't all reside in memory at once.
More from Infra
- Encrypted GPT-2 speedrun challenges FHE inference: best is 3.031 s/token, goal sub-1s — tensorqt · 2026-10-27
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27
- Stanford launches GPU-centric supercomputer Marlowe for data science and AI research — StanfordHAI · 2026-10-27
- NVIDIA Claims HGX Rubin NVL8 Delivers 10x More Tokens per Megawatt Than B200 — thursdai_pod · 2026-10-27
- Together lets multiple LoRA post-training experiments share the same GPU capacity — togethercompute · 2026-10-27
- Commoditization dilemma: either human labor or GPUs become the commodity — _AustinCalvert_ · 2026-10-27