hostely: One C++ binary to self-host containers and Metal-accelerated local LLMs on Mac mini
ayo_ham · x · 2026-09-06
A developer shipped hostely, a single C++ binary built over months of nights-only work for running real workloads on Mac minis and Apple Silicon machines.
Key features:
- Runs OCI containers via Apple's own container runtime (no Docker)
- Serves local GGUF LLMs at full Metal speed via bundled llama.cpp
- A dashboard reports real unified-memory headroom and refuses to load models that won't fit — the author notes no existing tool accounts for both GPU and containers sharing one memory pool
- Built-in HTTPS, domains, and tunnels, eliminating ngrok subscriptions
The repo is open source under MIT with a quickstart and smoke tests included.
Related event: hostely: Open-Source CLI Brings Containers and Local LLMs to Mac mini(3 posts)→
More from Infra
- Eric Hartford's fork unlocks P2P for RTX 3090/4090/5090 on NVIDIA's open kernel driver — QuixiAI · 2026-09-06
- DGX Spark driver fixes reclaim 64GB of 'missing' GPU memory and boost page faults 49x — QuixiAI · 2026-09-06
- Ollama cloud launches off-peak token pricing: DeepSeek-V4 at half price — ollama · 2026-09-06
- Ollama cloud full price list: $0.015 to $15 per million tokens across models — paw_lean · 2026-09-06
- AWS Gilroy 'Token Farm' data center takes shape next to a pepper farm — BenBajarin · 2026-09-06
- NInfer fork runs 555k-token context on a single RTX 5090 with custom NVFP4 KV cache and YaRN — Lumpy-Comedian-1027 · 2026-09-06