Local agent's 4 resident models ate 122GB VRAM; sleep daemon cut it to 43GB
ahstanin · reddit · 2026-09-12
The author runs a local voice+screen+tool agent with four models: Qwen3.8-27B (planning/tool calls), Nemotron (STT), Chatterbox (TTS) and Unlimited-OCR. Keeping them all resident in VRAM idled at 122GB. Key insight: agent loops are turn-based and sequential — OCR needs no memory during voice interaction, and the LLM can sleep while scripts run.
A Rust-based daemon controls sleep states across the cycle: LLM idle usage dropped from 87GB to 39GB, STT from 10GB to 267MiB, TTS from 18GB to 3GB, OCR to 422MiB — total idle memory from 122.3GB to 42.9GB. It drops scratch/KV allocations rather than cold-restarting, keeping wakeup latency under 200ms so voice back-and-forth feels instant. Running 4+ models on a single box becomes fully viable.
Related event: Rust Daemon Sleeps Idle Models, Cutting VRAM from 122GB to 43GB(3 posts)→
More from coding & agent
- Why working with AI agents all day is exhausting: delegation adds mental load, not removes it — bendee983 · 2026-09-12
- "Codex is my main interface to the system": user says agent made Linux admin effortless — mark_k · 2026-09-12
- Alchemy's distilled project builds unified agent-friendly SDKs for 60+ SaaS services — samgoodwin89 · 2026-09-12
- Kimi K2.7 RL recap: gains transfer to unseen benchmarks while steps drop ~35% — echen · 2026-09-12
- RL on just 1,700 tasks lifts Kimi K2.7 across five coding benchmarks — echen · 2026-09-12
- Expected Parrot's universal remote cache hits 30M entries to fix AI-agent reproducibility — soumitrashukla9 · 2026-09-12