744B Model Runs on 25GB Machine
alexcovo_eth · x · 2026-07-13
COLIBRI demonstrates a prototype that runs GLM-5.2 (744B parameters) on a consumer-grade machine with 25GB of RAM and no GPU.
The core idea is that the model doesn't need to keep all parameters resident in memory at once. Instead, it keeps a small portion in RAM and streams the rest from disk on-demand. While disk speed bottlenecks generation speed, it proves that running giant models doesn't strictly require massive VRAM. The project is open-source under Apache-2.0 and has garnered around 2.1k stars.
Related event: COLIBRI Runs 744B GLM-5.2 Model on 25GB RAM Without GPU(3 posts)→
More from Infra
- 80% of the DIY LLM inference hype posters have already quit — it's brutally hard systems work — abhijithneil · 2026-09-11
- Hugging Face's Ultra Scale Playbook: a free book on training LLMs on GPU clusters — mdancho84 · 2026-09-11
- Is inference latency becoming the biggest bottleneck for production AI agents? — Euphoric_Sea632 · 2026-09-11
- LLM Serving Metrics Thread: Why TPOT and Uptime Make or Break User Experience — abhijithneil · 2026-09-11
- PlanetScale launches sharded Postgres: 768 servers acting as one, 1PB scale — dhruv2038 · 2026-09-11
- Can a 7900 XTX 24GB run Qwen locally? Reddit seeks ROCm tok/s benchmarks — thenomadexplorerlife · 2026-09-11