MiniCPM-V 4.5 MoE reportedly hits 13+ tokens/s on iPhone with just 3GB memory
heyshrutimishra · x · 2026-10-16
The closing part of the MiniCPM-V 4.7 thread covers on-device performance:
- The series ships in two sizes: the 1B used in earlier demos and 35B-A3B, the series' first MoE model;
- Per the team's own figures, 35B-A3B reportedly runs at 13+ tokens/s using just 3GB of memory on an iPhone 17 Pro Max;
- On an iPad Pro 11" M4 (16GB), Lookahead Expert Scheduling pushes it from 2.97 to 14.92 tokens/s;
- The 1B's 4-bit GGUF is about 672MB; both run locally on Ollama and llama.cpp.
The author cautions these are team-reported numbers without full test conditions.
Related event: OpenBMB Releases MiniCPM-V 4.7 with First MoE Model Running on Phones(4 posts)→
More from Infra
- Reflection's 501B MoE Beam hits 463 tok/s on 8x GB200 with NVFP4 at day 0 — zhyncs42 · 2026-10-16
- Nemotron 3.5 Bolt runs 16 concurrent streams on a single DGX Spark — NVIDIAAI · 2026-10-16
- Why your computer can't store 0.2: an IEEE 754 primer — Franc0Fernand0 · 2026-10-16
- Sabaki: MIT-licensed typed decision router runs on CPU in milliseconds — Fox3000foxy · 2026-10-16
- BitRun v0.2 runs 8B ternary LLMs on CPU in 2GB RAM with live wattage telemetry — The_VibePreneur · 2026-10-16
- Solo-built Vulkan inference engine LaForge hits ComfyUI-level SD1.5 speed: 512px in 0.8s, 402MB container — Ichidan256 · 2026-10-16