Running GLM 5.2 on a Low-Spec Computer
vforno · hn · 2026-07-09
The author shares their journey of running **GLM 5.2** on a low-spec computer, naming the project **Colibrì**. Driven by curiosity to see if the model could run on a standard PC without exhausting memory, they leveraged agents for continuous engineering experiments. Key approaches include: - Converting the model to **int4**, exploring MTP usage, and attempting long-context DSA. - Leveraging the model's **MoE** architecture: keeping dense components in memory while streaming routed experts from disk on demand. - Introducing a **per-layer LRU cache**, an optional hot-store, and relying on the OS page cache as a "free L2 cache." The author claims they eventually interacted with the model on a machine with **32GB RAM**. Although cold starts were slow (around **0.1 tok/s**), the goal was simply to "make it run." The project itself is a single-file engine of about **1300 lines of C code**, with no dependencies on BLAS, Python, or GPUs.
More from Infra
- Local AI may pay back in 6–7 years and cut long-term costs by 30–40% — DavidLinthicum · 2026-07-21
- TSMC reportedly plans up to 10% chipmaking price hikes in 2027 — kimmonismus · 2026-07-21
- More open models and llama.cpp updates are coming, says Merve Noyan — mervenoyann · 2026-07-21
- Why adding a second LLM provider breaks more than the API surface — Ok_Extension6373 · 2026-07-21
- UK AI datacentres face backlash over heat, noise and land use — nordicinst · 2026-07-21
- Fluidstack raises $830M at $7.5B valuation as Anthropic backs a $50B compute buildout — rohanpaul_ai · 2026-07-21