Measured VRAM use of Qwen3.5 9B and Gemma 4 E4B on a 3060 12GB, plus a free fit calculator
sdatta11 · reddit · 2026-11-20
The author measured real VRAM use with LM Studio (llama.cpp CUDA, Q4KM, full GPU offload) and built a free browser-based calculator (within 1% of measurements).
Measured data (4K/16K/32K ctx):
- Gemma 4 E4B: 4.51→5.01 GB, 74 tok/s
- Qwen3.5 9B: 6.66→7.63 GB, 54 tok/s
- Qwen3.8 27B: doesn't fit — 9.5 GB on GPU, rest in RAM, 5.4 tok/s
Two findings:
- Hybrid models barely grow with context: Qwen3.5 keeps KV cache on only 8 of 32 layers (rest linear attention), 0.034 GB per 1K tokens — half of a similar-size Qwen2.5-style 7B, 10× less than Gemma 2 9B.
- Gemma 4 E4B is nominally 8B params, but 2.8B are per-layer embeddings kept in system RAM by llama.cpp, so it uses less VRAM than its size suggests.
The calculator covers 54 desktop/laptop GPUs and 40 models at Q4/Q5/Q8/FP16 (weights + KV cache for full-attention layers + 0.6 GB runtime), flags "runs on GPU / spills to RAM / won't run", and outputs the ollama command for the biggest model that fits. No sign-up, runs entirely in-browser.
More from Infra
- Gemini Ultra's $100 plan includes 1000 monthly Colab credits—about 55 H100 hours — iskander · 2026-11-22
- Hy team pushes back on ultra-sparse MoE: sparsity recovers efficiency, not quality — douglovefishing12 · 2026-11-22
- SpaceX lenders push back on $40B of AI chips launched into space — flngr · 2026-11-22
- Homie: an offline Mac notch voice assistant running Qwen 3.5 4B locally — KungFuSaifooo · 2026-11-22
- AMD and PyTorch ship a playbook to run GPT-OSS-20B inference locally with ROCm — PyTorch · 2026-11-22
- Microsoft's full Outlook outage PIR confirms AI tooling wasn't to blame — aronchick · 2026-11-22