MiniCPM-V4.7: 35B MoE multimodal model runs on phones with 3GB RAM at 30 tokens/s
机器之心 · wechat · 2026-11-09
ModelBest (面壁智能) and OpenBMB open-sourced MiniCPM-V4.7. The 35B-A3B MoE multimodal model runs on-device with about 3GB of RAM, exceeding 30 tokens/s on iPhone 18 Pro Max, with capability the team says surpasses 2023's cloud flagship GPT-4V.
- Core infra technique: Lookahead Expert Scheduling (LES) stores expert weights in flash storage and predicts the next layer's experts using the stability of adjacent-layer expert selection enabled by residual connections, overlapping loading with compute with negligible accuracy loss.
- Model-side gains: aggressive visual compression (up to 96x), low-cost RLPR rewards, and anti-forgetting training boost capability density; the 1B variant targets sub-$150 phones.
- Co-chief scientist Yao Yuan frames model/hardware/infra as three flywheels whose multiplicative gains could reach 5-8x, predicts GPT-6-class intelligence on phones within 2 years, and expects 90% of bounded everyday AI needs to migrate from cloud to device.
- Full open-source release includes weights, on-device inference code, app packages, fine-tuning recipes, and support for AMD/Apple/Qualcomm platforms plus multiple domestic AI chips.
More from Infra
- Batch APIs: the 50% LLM discount only pays off after model choice and prompt caching — arpit_bhayani · 2026-11-13
- RTX Pro 6000 bricks mid-Qwen run, owner jokes it "died of a broken heart" over Jensen news — wine_dark · 2026-11-13
- Bittensor's Chutes Handles 256 Concurrent Requests on a Single RTX 5090 at ~18k tok/s — markjeffrey · 2026-11-13
- Google expands TPU v8 lineup: 8tx cuts latency 2x, 8i Max doubles silicon per rack — Beth_Kindig · 2026-11-13
- Horace He to share two mental models for reasoning about inference at SGLang Summit — cHHillee · 2026-11-13
- Used A100 prices fall another 20-30% as sellers rush ahead of a collapse — Bascilian · 2026-11-13