He built a €18k server with 768GB VRAM—now 2-trillion-parameter open models may leave it behind
myreala · reddit · 2026-09-04
A Reddit user spent €18k building an EPYC server with twelve 64GB cards (768GB VRAM total) plus 256GB RAM to run frontier open-source models locally. Now he fears he's already obsolete: next-gen open frontier models look likely to hit 2 trillion parameters or more.
In his view only GLM 5.3 still fits, and it will lag once Astra ships; GLM 6 could be two to three times larger, while Qwen-max, Kimi, and DeepSeek V4 Pro are already too big to consider.
He's weighing whether to sell the excess GPUs and settle for flash models on fewer cards. He had hoped to build a business on it, but admits the same business could probably run on a flash model with more engineering effort—a textbook dilemma for hobbyist-scale compute in an era of rapidly ballooning models.
More from Infra
- PostgreSQL 19 slips to November as LLM-assisted code review floods committers with edge cases — sanjaykalra · 2026-10-21
- Decision Models' routing classifier hits AntSeed, cutting inference cost to 24% of always-Opus — airesearch12 · 2026-10-21
- Eon cuts engineer onboarding from 2 days to 2 minutes with an MCP Gateway on Bedrock AgentCore — AWS ML Blog · 2026-10-20
- a16z's Ben Horowitz says AI buildout isn't the 90s bubble — an investor pushes back — henloitsjoyce · 2026-10-20
- Micron and Nvidia drive ~36% of S&P 500's Q3 earnings growth — Beth_Kindig · 2026-10-20
- NVIDIA lands in Europe: Jensen Huang and German chancellor discuss industrial AI at GTC Berlin — nvidia · 2026-10-20