744B GLM-5.2 MoE Model Runs Locally on 25GB RAM

The tech community is actively discussing a method to run the massive 744B GLM-5.2 model locally on consumer machines with only 25GB of RAM and no GPU. This breakthrough attempt challenges the absolute reliance on high-end hardware for massive models and has attracted significant attention.

Core Execution Mechanism

This setup's ability to run on extremely low memory relies on exploiting MoE (Mixture of Experts) architecture and low-level optimizations. According to developers, while GLM-5.2 has 744B total parameters, it only activates about 40B parameters per token. Therefore, systems like the Colibrì project keep the frequently used dense tensors in RAM while offloading the massive expert weights to NVMe storage. The system also employs an LRU cache mechanism and KV cache compression to further reduce the memory footprint.

Limitations and Trade-offs

Although booting a 744B model on a 25GB RAM device is remarkable, the current approach involves significant compromises. Multiple authors point out that this method of aggressively minimizing memory usage results in very slow inference speeds. It currently serves more as a proof-of-concept for technical feasibility rather than a practical, production-ready solution.

2026-07-10 ~ 2026-07-11 · 5 related posts

• GLM-5.2 Runs on 25GB RAM Machines — yogthos · 2026-07-10
• [source] 744B GLM Runs on 25GB RAM — rohanpaul_ai · 2026-07-10
• How GLM-5.2 Runs Locally Explained — rohanpaul_ai · 2026-07-10
• [source] 744B Model Runs on 25GB RAM — techNmak · 2026-07-11
• [source] 744B Model Runs Locally on a Laptop — WesEklund · 2026-07-11