Who Can Actually Run 500B+ Models Locally? Data Points Wanted
HankYeomans · x · 2026-10-31
nettermina asks owners of enough Spark devices and Mac Ultras to run 500B+ parameter models to share decode and especially prefill numbers, along with model and quantization details, preferably at 32k/64k/128k context — noting such firsthand data points are rarely seen.
More from Infra
- Workshop: Build a media agent on CPU, call image and TTS models as MCP tools — Al_Grigor · 2026-11-01
- Amazon reportedly junking 2-3-year-old H100/H200 GPUs while books still assume 6-year depreciation — SumitGup · 2026-10-31
- Is a used 64-core Threadripper PRO 3995WX worth it for a multi-GPU LLM server? — Ammoryyy · 2026-10-31
- Altman says owning RAM is wasteful; netizen flexes 64GB running Hello World — SuB8u · 2026-10-31
- xAI-adjacent concept: lunar mass drivers to launch 1000+ TW/year of AI satellites — XFreeze · 2026-10-31
- The AI race is now an infrastructure race: Blackwell packs 208B transistors and 1.8TB/s NVLink — Olivier__OG · 2026-10-31