Andrew Chen's homelab for local AI: 5090 eGPU, dual DGX Spark and a routing plugin
andrewchen · x · 2026-09-14
a16z partner Andrew Chen shared his full homelab setup for local AI experimentation:
- Local inference: a Framework Desktop AI Max+ 395 running Hermes, plus a 5090 eGPU for Qwen 27B at 150+ tok/s; two DGX Spark boxes run DeepSeek v4 Flash for slower but stronger background batch jobs like cron tasks and long dev builds.
- Routing: a homegrown plugin uses Arch-Router, a small low-TTFT model, to decide whether requests stay local or upgrade to cloud frontier models — currently 60-70% local, aiming for 100%.
- Extras: a Pi 5 for monitoring, a Mac mini dev box running Herdr and ohmypi/codex/claude, all in a 10" DeskPi mini rack.
He admits none of this is strictly necessary and warns he regrets the eGPU purchase — wouldn't recommend it.
More from coding & agent
- Hugo Bowne launches Master Agentic Data Science workshop with PyMC's Thomas Wiecki — hugobowne · 2026-09-14
- DeepSeek Harness hailed as the best multi-agent harness outside the labs — bookwormengr · 2026-09-14
- pi-vcc vs pi-blackhole: Sub-second Lossless Context Compaction or 3-LLM Recall? — mailto_devnull · 2026-09-14
- Homebrew 7.0.0 ships with native macOS app, sandboxing and vulnerability checks — MABUwaAI · 2026-09-14
- CS professor bans coding agents on assignments, students' reaction becomes a meme — CSProfKGD · 2026-09-14
- Teknium Spotlights Hermes Telephony Skill Turning Agents Into Phone Operators — Teknium · 2026-09-14