Running a 26B Model on an Old Xeon
neomindryan · hn · 2026-07-15
This article demonstrates running Gemma 4 26B on a 13-year-old Xeon machine without a GPU, achieving a speed of about 5 tokens/s.
The focus is on viable inference on low-end hardware. The author uses this post to illustrate that running large models doesn't strictly require modern GPUs.
More from Infra
- AI demand reshapes cloud computing: scarce capacity and long-term deals replace on-demand — DavidLinthicum · 2026-10-26
- NVIDIA Plans ~$2,000 Per GPU Per Year Software Fee for Running AI Agents — LinusEkenstam · 2026-10-26
- 12 DevOps concepts to learn in order, from Linux to SRE — _jaydeepkarale · 2026-10-26
- Killed our Vector DB for SQLite BM25 and wrote a 3.6ns Rust sentinel: lessons from multi-agent loops — OpenComplaint524 · 2026-10-26
- Gemma 4 26B-A4B runs offline on a 16 GB Mac: 33/40 on TypeScript tasks at 31 tok/s — Mission-Sir-1614 · 2026-10-26
- How thousands of GPUs train one AI: a breakdown of 5D parallelism — jbhuang0604 · 2026-10-26