PrismML releases open-weight ternary Bonsai 2 models: 2B at 0.43GB, 90 tok/s on iPhone
gajesh · x · 2026-10-16
PrismML launched Bonsai 2, open-weight ternary models in 2B, 4B and 9B sizes based on Qwen3.5, built for on-device use:
- Tiny footprints: 2B is just 0.43GB, 4B is 0.95GB, 9B is 2.0GB
- On-device speed: the 2B hits 90 tokens/sec on iPhone 17 Pro Max; the 9B does reasoning, vision and tool calling locally at 170+ tokens/sec at launch
- Practical focus: instruction following, avoiding repetitive loops, mobile tool calling; the 9B uses 40% fewer tokens on standard instruction-following and everyday math tasks
Two speed-run tracks are open (Apple Silicon via mlx.fast, NVIDIA DGX Spark via YukonResearch); a solver already made the 9B run 13% faster on Mac with speculative decoding (decode 146.5→171.1 tok/s), and every new record is open for anyone to build on.
Related event: PrismML open-sources ternary Bonsai 2 models, 2B runs in browser at 0.43GB(8 posts)→
More from Infra
- First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16