Bonsai 2 ternary models tested: 4B is the sweet spot, but 27B unexpectedly fumbles file writes
cephaloform · x · 2026-10-16
Hands-on results for the Bonsai 2 ternary family (2B/4B/9B), with charts benchmarking all sizes against their teacher, Qwen3.8-27B, across seven tests:
- 4B is the sweet spot: 1.06 GB at 547 tok/s, within 5 points of the teacher on 2 of 7 tests, HumanEval 90.
- 9B is exceptional for its size (2.24 GB, 384 tok/s), HumanEval 94, GSM8K 97.
- 2B is just 483 MB but runs at 878 tok/s with consistent strengths.
- Thinking modes are now a simple on/off, eliminating the excessive capping of the original.
- Notable regression: the 27B scores only 51 on Payload (writefile calls) — 47 of its 59 misses wrote a placeholder instead of text.
More from Models
- Nolla Health Post-Trains Nemotron 3.5 VL: Visual Diagnosis Jumps from 35% to 66.8% — LoganGrasby · 2026-10-16
- Dev argues coding models now beat nearly all engineers in every branch of software — zack_overflow · 2026-10-16
- Prediction: Chinese frontier models could match GPT-6 by March 2027 — teortaxesTex · 2026-10-16
- Liquid AI's d1-ultra, #1 on Artificial Analysis Decision Index, lands on Vercel — JosephJacks_ · 2026-10-16
- Weekly AI roundup: Claude Haiku 5.5 cuts prices ~90%, Gemini 4 Argon output jumps to 1M tokens — DeepLearningAI · 2026-10-16
- Tencent Hunyuan says reported agent activity came from third-party model evals, not deliberate targeting — TencentHunyuan · 2026-10-16