Local agent's 4 resident models ate 122GB VRAM; sleep daemon cut it to 43GB
ahstanin · reddit · 2026-09-12
The author runs a local voice+screen+tool agent with four models: Qwen3.8-27B (planning/tool calls), Nemotron (STT), Chatterbox (TTS) and Unlimited-OCR. Keeping them all resident in VRAM idled at 122GB. Key insight: agent loops are turn-based and sequential — OCR needs no memory during voice interaction, and the LLM can sleep while scripts run.
A Rust-based daemon controls sleep states across the cycle: LLM idle usage dropped from 87GB to 39GB, STT from 10GB to 267MiB, TTS from 18GB to 3GB, OCR to 422MiB — total idle memory from 122.3GB to 42.9GB. It drops scratch/KV allocations rather than cold-restarting, keeping wakeup latency under 200ms so voice back-and-forth feels instant. Running 4+ models on a single box becomes fully viable.
Related event: Rust Daemon Sleeps Idle Models, Cutting VRAM from 122GB to 43GB(3 posts)→
More from coding & agent
- One-person companies are about the org chart, not headcount: humans judge, agents execute — ciguleva · 2026-10-28
- OATs: open-source toolkit turns everyday objects into personality-bearing agents — Stefania_druga · 2026-10-28
- Dev: local coding dropped from 100% to 5% in a year as cloud agents take over — bytebot · 2026-10-28
- Open-sourced 501(c)(FREE): agent-built free back office for US nonprofits — gralston · 2026-10-28
- Grok Bot 1.0.0 early build reveals Shared Bots, service accounts and team workflows — XFreeze · 2026-10-28
- Anne Griffin hosts 20-person SF salon on the product decisions behind AI agents — annetgriffin · 2026-10-28