Open-Sourced Recipe: Running a 27B Local Agent 24/7 on a Single RTX 5090
max_paperclips · x · 2026-08-05
Developer witcheer released the 4th Hermes Recipe, detailing how to deploy a 27B AI agent fully locally and 24/7 on a single consumer GPU (like an RTX 5090 32GB).
The setup has been running stably on an Ubuntu server, featuring:
- System Services: systemd units for boot-persistent model inference and the agent gateway.
- Identity & Memory: Markdown templates (SOUL/USER/MEMORY) to steer model behavior, with a memory vault supporting CPU-based semantic search.
- Skill Pack: Includes a daily morning brief and a local card renderer for headless boxes.
- Resource Scheduling: A drain/restore pattern allowing users to temporarily borrow the GPU for training runs and safely restore the agent afterward.
- Security Posture: Zero inbound ports via long-polling, strict platform allowlists, and scoped sudoers rules.
More from coding & agent
- Self-Improving Agents Boost B200 Inference Throughput by 16% Without Losing Accuracy — yisongyue · 2026-08-05
- DeepSeek V4 Flash Offered at 90% Off on Vercel, Touted as Opus 4 Rival — cramforce · 2026-08-05
- The Bottleneck of AI Coding Isn't the Model, It's Your CI Pipeline — hichaelmart · 2026-08-05
- Developer Tests OMP Coding Agent: Fixes Local LLM Inference Lag in One Prompt — Sentdex · 2026-08-05
- Dev Releases Open-Source Agentic OS for Robotics with MIT License — mimi10v3 · 2026-08-05
- Developers Spot Mainstream LLMs Suddenly Obsessed with 'Smoke Testing' — TokenRingAI · 2026-08-05