Out of Claude and Codex tokens, dev spins up local vLLM serving Qwen FP4 model for agents
alfcnz · x · 2026-11-09
Developer alfcnz ran out of Claude and Codex tokens, so he now serves his own "intelligence" on demand: vLLM hosting nvidia/Qwen3.8-Flash-Next-FP4-Local on a local GPU. Screenshots show the full stack — power/VRAM monitoring via btop, a homegrown 'spark' CLI to start/stop the server and open an API tunnel over Tailscale — currently generating personal lessons and powering a hands-on assistant. A representative case of falling back to local inference when subscription quotas run dry.
More from coding & agent
- Revolut runs coding agents for 2,000 engineers at 1/10th cost with open models — zainhas · 2026-11-09
- Future CEO of agentic orgs will be a system designer, not a McKinsey alum — GregCook2011 · 2026-11-09
- When humans become the bottleneck, lift the loop: System 1 steers, agents deliver — dotey · 2026-11-09
- Run Claude Code on Your iPhone: Rex + Tailscale Setup Guide — leslysandra · 2026-11-09
- Ex-OpenAI RLHF lead Diogo's north star: make intelligence as boring as databases — schwentker · 2026-11-09
- Grok Ghost demo shows autonomous computer-use agent driven by screen OCR — Aiden_Tech_Ai · 2026-11-09