744B Model Runs on 25GB Machine
alexcovo_eth · x · 2026-07-13
COLIBRI demonstrates a prototype that runs GLM-5.2 (744B parameters) on a consumer-grade machine with 25GB of RAM and no GPU.
The core idea is that the model doesn't need to keep all parameters resident in memory at once. Instead, it keeps a small portion in RAM and streams the rest from disk on-demand. While disk speed bottlenecks generation speed, it proves that running giant models doesn't strictly require massive VRAM. The project is open-source under Apache-2.0 and has garnered around 2.1k stars.
Related event: COLIBRI Runs 744B GLM-5.2 Model on 25GB RAM Without GPU(3 posts)→
More from Infra
- Encrypted GPT-2 speedrun challenges FHE inference: best is 3.031 s/token, goal sub-1s — tensorqt · 2026-10-27
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27
- NVIDIA claims HGX Rubin NVL8 delivers 10x tokens per megawatt vs B200 — thursdai_pod · 2026-10-27
- NVIDIA Claims HGX Rubin NVL8 Delivers 10x More Tokens per Megawatt Than B200 — thursdai_pod · 2026-10-27
- Together lets multiple LoRA post-training experiments share the same GPU capacity — togethercompute · 2026-10-27
- Commoditization dilemma: either human labor or GPUs become the commodity — _AustinCalvert_ · 2026-10-27