Running MiniCPM5-2B as a local agent on M4 16GB: trimming, tool-call adapter and benchmarks

面壁智能 · wechat · 2026-09-24

A developer post on ModelBest's WeChat account details building a lightweight local personal assistant with MiniCPM5-2B (8-bit MLX quantization) on an M4 16GB Mac, using the open-source OpenHanako/HanaAgent framework with cloud Kimi handling vision.

Key fixes and modifications

First tool-call tests failed with "multi-task concurrency collapse": XML tags mixed with prose, hallucinated file paths, and aborted execution — diagnosed as a 2B model's attention budget failing under simultaneous system-prompt overhead, strict tool-format alignment, and open-ended generation.
Source-level pruning of OpenHanako: removed ComputerUse, extra messaging platforms, multi-agent collaboration, image/video generation, and auto-update code, shrinking the tool surface to 11 essentials; Skills now inject only name/description and read on demand.
A custom XML tool adapter converts MiniCPM's function XML into standard toolcalls with streaming boundary detection and JSON Schema validation; system prompt reduced to 5 hard rules.
Concrete settings: context 16384, temperature 0.2, topP 0.9, topK 40, repetitionPenalty 1.05; MLX server capped to single-request decode with 3GB prompt cache.

Evaluations: seasonal prose, story continuation, a 2026 postgraduate math-exam problem, and coding tasks all passed cleanly — strong results for a local 2B model. Both projects are open-sourced on GitHub.

Original post →

Running MiniCPM5-2B as a local agent on M4 16GB: trimming, tool-call adapter and benchmarks

More from coding & agent