Open-source zxLLM predicts LLM VRAM usage & KV-cache needs with high precision
Capable_Item_5918 · reddit · 2026-08-17
A developer released zxLLM, an open-source tool designed to accurately predict VRAM usage and KV-cache requirements for local LLMs. It supports vLLM, SGLang, and llama.cpp, automatically detecting architectural nuances like GQA and querying local GPU states via nvidia-smi. Tested on models like Llama 3, Qwen 2.5, and DeepSeek, it achieves a low error rate (0.04% - 1.8%) and runs with zero external dependencies.
More from coding & agent
- Obscure Performance Fix Cuts CPU Usage by 80% — DanielLockyer · 2026-08-17
- Switching to direct DB connections cut P95 latency from 480ms to 80ms — DanielLockyer · 2026-08-17
- OpenAI embraces x402 for agent commerce; author flags whitelist scalability issue — MountainAssignment36 · 2026-08-17
- Wasting 2M context? Google's guide suggests a different structure for Gemini — thisdudelikesAI · 2026-08-17
- Luna Subagents Could Verify Game Wikis, Codex Reads Codebases — Angaisb_ · 2026-08-17
- Writing C code manually with AI autocomplete beats AI generation — tetsuoai · 2026-08-17