PrismML's open-source Ternary Bonsai 2 squeezes a 9B VLM into ~2GB
lmoroney · x · 2026-10-17
PrismML released the Ternary Bonsai 2 family under Apache 2.0 on Oct 15, compressing Qwen3.5-based vision-language models via ternary weights (-1/0/+1 with group-wise scales).
- New 2B/4B/9B models join the existing 27B; footprints are 0.43GB, 0.95GB, and 2GB — over 9x smaller than full precision.
- Runs on NVIDIA GPUs via llama.cpp and on Mac/iPhone/iPad via MLX; vendor-reported up to 179 tok/s for the 9B on an M5 Max Pro and 34 on an iPhone 18 Pro Max.
- Reported benchmark retention vs Qwen3.5 baselines is roughly 92–100% for the 9B — treat as vendor figures until tested.
- Practical caveat: ternary GGUF packings need PrismML's llama.cpp fork; upstream llama.cpp only loads the Q20 file.
The author also suggests a classroom exercise: run the 4B on a laptop, then find which skill group dropped most in the benchmark table — a ready-made discussion on what compression costs.
Related event: PrismML open-sources ternary Bonsai 2 models, 9B at just 2GB(10 posts)→
More from Infra
- Fund Managers Name AI Hyperscaler Capex the Top Systemic Credit Risk — SuB8u · 2026-10-17
- Vercel's tsc-ts ports tsgo back to TypeScript, runs 2-4x faster than tsgo — cramforce · 2026-10-17
- Kevin O'Leary refiles Utah data center plan using 558M gallons of water a year — Zachly · 2026-10-17
- Wishing the RAM shortage makes school Chromebooks unaffordable — AI spillover hits education — RachelVT42 · 2026-10-17
- Compute Labs launches financing platform matching GPU operators with capital — paigeinsf · 2026-10-17
- Google researcher explains why 5:1 interleaved SWA won: XLA compile times and TPU constraints — teortaxesTex · 2026-10-17