Why 128GB RAM matters for local AI on a 5090: three-tier offloading explained
QuixiAI · x · 2026-09-06
QuixiAI explains the reference systems behind its Local AI Lab setup, restricted to hardware buyable at retail today: a 5090 with 128GB RAM ($7500) and a DGX Spark ($4700); a Mac Studio M3 Ultra 128GB was dropped since Apple stopped selling it.
The key insight: large RAM matters on the 5090 because SlimServe uses three offload tiers—GPU VRAM, CPU RAM, and NVMe—while unified-memory machines like DGX Spark and Mac Studio lack a CPU offload tier, leaving only VRAM plus NVMe.
Related event: QuixiAI Shares Reference Config for a Local AI Lab(3 posts)→
More from Infra
- Dwarkesh publishes 48 flashcards on advanced chip manufacturing to accompany Intel episode — calabi_and_yau · 2026-10-23
- Kazakhstan expands NVIDIA-powered compute as Firebird plans local AI lab — nvidia · 2026-10-23
- Jensen Huang visits Astana, joins Kazakhstan's AI minister to discuss nation's AI future — nvidia · 2026-10-23
- FP16 vs FP32 matmul on RTX 4060: 3.5x speedup at large sizes, slower at 128x128 — cneuralnetwork · 2026-10-23
- BaseQuant V1 quants for agentic Qwen3.8-27B hit 17-34% lower KL than Unsloth — nightowlsleeping · 2026-10-23
- PhD students in some countries get B200s on demand, unlike their US peers — HildeKuehne · 2026-10-23