27B Ternary Model Runs Within 10GB Memory
tcarambat · reddit · 2026-07-15
The post introduces PrismML's newly released Bonsai 27B: a Qwen-series model scaling the Ternary / BitNet approach to 27B parameters.
Key highlights include:
- Capable of running a 27B model on around 10GB of memory, with performance near fp16 precision at a 32K context length;
- Delivers significantly better results compared to similarly sized low-bit quantized versions, aiming to make large local models truly usable;
- They also demoed the model taking control of an entire computer via OpenComputer to conduct browser research and generate HTML reports.
The post also mentions unconfirmed capabilities and ecosystem details, such as dFlash, MTP support, 256K context, multimodal input, alongside ongoing llama.cpp and MLX branches. The author views this release as more significant than some higher-profile model updates, suggesting it marks a new milestone for local and edge-side open-source models.
Related event: Bonsai 27B: The first 27B model that runs on phones(15 posts)→
More from Infra
- DeepSpeed shows AutoEP/AutoTP/AutoSP parallelism gains, hosts office hours — PyTorch · 2026-10-27
- Cloudflare Durable Objects validated as top architecture for AI agent WebSocket servers — tristanbob · 2026-10-27
- Gary Marcus: With Q3 data added, Anthropic's compute-efficiency growth shows deceleration — GaryMarcus · 2026-10-27
- SpaceX locks in $40B GPU financing deal with Apollo, the largest ever committed — TechCrunch AI · 2026-10-27
- Inference is just Kubernetes with a multi-trillion-dollar market, and the communities overlap — matt_slotnick · 2026-10-27
- vLLM eyes Rust migration as inference engines race to rewrite the stack — AccBalanced · 2026-10-27