GLM-130B Training Cost About $600K: Two Months on 96 A100s
ccerrato147 · x · 2026-11-09
GLM-130B was deliberately smaller — 130B parameters fitting on one A100 server. Training began in January 2022 on 96 A100s, repeatedly failing on memory, gradients and broken hardware. The successful run took two months and roughly $600K of compute, then the weights were published.
Related event: Anatomy of Zhipu's Open-Source Gamble: $110M Revenue, Volatile Valuation(13 posts)→
More from Infra
- Vork: an agent-native 3D engine that renders 24x faster than Blender with a 2,500-token API — ravisparikh · 2026-11-09
- Akamai Edges Cloudflare 78-76: A 13x Sales-Multiple Gap Between Two Edge Giants — pdamodaran · 2026-11-09
- Out of Claude and Codex tokens, dev spins up local vLLM serving Qwen FP4 model for agents — alfcnz · 2026-11-09
- AGI House's personal agents build day pushes local inference and intelligence-per-watt — agihouse_org · 2026-11-09
- Agentic AI is orchestration of model calls; datacenter limits may push edge models — DavidLinthicum · 2026-11-09
- "Bro got distilled": meme pairs Jensen Huang photo with Huawei's Nvidia chip challenge — StephanSturges · 2026-11-09