Andrew Chen's homelab for local AI: 5090 eGPU, dual DGX Spark and a routing plugin
andrewchen · x · 2026-09-14
a16z partner Andrew Chen shared his full homelab setup for local AI experimentation:
- Local inference: a Framework Desktop AI Max+ 395 running Hermes, plus a 5090 eGPU for Qwen 27B at 150+ tok/s; two DGX Spark boxes run DeepSeek v4 Flash for slower but stronger background batch jobs like cron tasks and long dev builds.
- Routing: a homegrown plugin uses Arch-Router, a small low-TTFT model, to decide whether requests stay local or upgrade to cloud frontier models — currently 60-70% local, aiming for 100%.
- Extras: a Pi 5 for monitoring, a Mac mini dev box running Herdr and ohmypi/codex/claude, all in a 10" DeskPi mini rack.
He admits none of this is strictly necessary and warns he regrets the eGPU purchase — wouldn't recommend it.
More from coding & agent
- Element Forge: open-source editable AI VFX for real footage in ComfyUI — zitroffell · 2026-10-30
- Gym-Anything Adds Audit Agents That Try to Break Simulations With Legal Robot Actions — wellecks · 2026-10-30
- CMU's Gym-Anything Auto-Builds 200+ Agent Environments from Any Software, Selected by GDP — wellecks · 2026-10-30
- durable-object-migrator: zero-downtime exports for Cloudflare Durable Objects — samgoodwin89 · 2026-10-30
- This 5-question interview prompt makes Grok design your first two custom bots in minutes — nateliason · 2026-10-30
- Cursor and Harvey swap general models for task-specific brains, but most niches can't afford to — MartinGTobias · 2026-10-30