hf-mem v1.0.0 estimates LLM inference memory without downloading weights

SergioPaniego · x · 2026-11-17

Hugging Face's hf-mem v1.0.0 estimates inference memory requirements (weights, KV cache, runtime overhead) without pulling model weights, supporting Safetensors, GGUF and ONNX via HTTP range requests, with only httpx2 as a dependency. Run e.g. uvx [email protected] Qwen/Qwen3.8-27B --max-model-len 32768 --gpu-memory 96 for a per-component breakdown — useful for humans and agents alike.

Original post →

More from coding & agent

coding & agent channel →