MiniCPM-V 4.7 runs locally at 100 tok/s using only ~3 GB of memory
NielsRogge · x · 2026-10-17
Niels Rogge demonstrates running OpenBMB's new multimodal MiniCPM-V 4.7 locally with llama.cpp at 100 tokens/second using only 3 GB of memory, with a full walkthrough thread via llama.cpp's web UI. The collection ships 35B-A3B and 1B variants, targeting on-device multimodal AI.
Related event: OpenBMB Releases MiniCPM-V 4.7: On-Device Multimodal MoE Taking on Qwen3.6(5 posts)→
More from Infra
- Custom VBIOS loader lets you edit RTX 50/40/30 power limits and voltages without flashing — panchovix · 2026-10-17
- Perplexity Flash matches Wald's accuracy at nearly 2x speed in local agent-decision benchmark — WolframRvnwlf · 2026-10-17
- Mac LLM Calculator tells you which Apple Silicon runs which model, and how fast — shauntrennery · 2026-10-17
- llama.cpp PR #30517 Delivers Big CUDA Speedup for Low-Bit Q2_K/Q3_K Quantization — jacek2023 · 2026-10-17
- India has only ~30,000 GPUs nationwide, 10-20x short of frontier model needs — CatAstro_Piyush · 2026-10-17
- Power-management chips: the invisible layer deciding AI data center energy efficiency — Olivier__OG · 2026-10-17