PrismML ships Bonsai 2 ternary models: 9B fits in 2GB and beats Qwen3.5 on instruction following
calabi_and_yau · x · 2026-10-16
PrismML released Bonsai 2, a family of ternary models based on Qwen3.5 in 2B/4B/9B sizes weighing just 0.43GB, 0.95GB and 2.0GB, targeting on-device local assistants on phones, PCs and wearables.
Key points
- Focus on practical local assistant behavior: instruction following, avoiding repetitive loops, mobile tool calling, minimal output
- The 9B uses 40% fewer tokens on standard instruction-following and everyday math tasks
- Benchmarks show Bonsai beats the corresponding Qwen3.5 originals on instruction following and one-shot coding at every size, with close math scores but lower knowledge/reasoning and vision
- Apache 2.0 licensed; the family spans four sizes from 2B to 27B, positioned as a foundation for agentic local apps
Related event: PrismML Open-Sources Ternary Bonsai 2 Models for On-Device Use(6 posts)→
More from Infra
- New NVFP4 trick: compressing block scales saves 45-48% per expert matrix, bit-exact — TheZachMueller · 2026-10-16
- AWS extends Claude Sonnet 3/3.5/3.6 EOL on Bedrock to November 13, 2026 — repligate · 2026-10-16
- After Apple and Meta CEOs tout Mac Silicon inference, a $10B distributed network idea resurfaces — gajesh · 2026-10-16
- NVIDIA's 74B Nemotron 3.5 Bolt, top US open-weight model, now runs locally on Mac — SumitGup · 2026-10-16
- Meta open-weights 816B Muse Spark Open, Apple tunes it for MLX on Mac — rohanpaul_ai · 2026-10-16
- Running Nemotron-3.5-Bolt-74B MoE on a single RTX 3060 with 12GB VRAM — cephaloform · 2026-10-16