Quartermaster: open-source local AI platform that auto-configures models by free VRAM
OneMoreName1 · reddit · 2026-09-04
Quartermaster is an open-source (MIT) local AI platform, forked from llama-swap, aiming for ease of use without sacrificing customizability.
Key features:
- Point it at your models folder and it configures things: reads GGUF headers, measures actual free VRAM, and works out context length, GPU offload, CPU/MoE split and KV cache size per model — all editable if you disagree.
- Multiple backends: llama.cpp (Vulkan/CUDA/ROCm/CPU builds downloaded and kept updated), stable-diffusion.cpp (SD/SDXL/Flux/Qwen-Image/LoRAs/upscaling), vLLM, plus any executable you register (e.g. TTS, ikllama).
- Everything sits behind one OpenAI-compatible API on one port with a single scheduler, so models swap in/out without fighting for VRAM.
- Built-in chat playground with web search, plus a Hugging Face browser to search models, pick a quant and download directly.
More from coding & agent
- Theo: Astra is world-class at everything except frontends and mergeable code — adonis_singh · 2026-09-04
- Multiplayer World Simulations: Realtime LLM Plus h3 Lets Each Player Control Only Their Own Character — realimposter · 2026-09-04
- Context7 team launches Docs7 to serve agent-readable docs with WebMCP and skills support — dinasaur_404 · 2026-09-04
- Replit CEO: GPT-6 is a major capability jump, launching on Replit very soon — amasad · 2026-09-04
- Nous Research brings one-click local model setup to Hermes Agent on NVIDIA systems — markjeffrey · 2026-09-04
- GPT-6 Astra early tests: agent edits video, retouches photos, builds 4K Blender renders — gabrielchua · 2026-09-04