Tracking 1-bit/Ternary models: Bonsai 27B updates & Maple speed boosts
pmttyji · reddit · 2026-08-19
A roundup of low-bit model updates:
- Bonsai: 27B CUDA/Vulkan support merged to llama.cpp mainline; CUDA optimizations yield +15-40% tok/s.
- Maple-Preview: 20B-A1B hits 200+ t/s on Mac Mini M4.
- Mach-1: 35B model achieves 120 t/s on consumer laptops.
- Also tracks Neutrino-8B, Pestle-27B (medical), and various image model quantizations.
More from Infra
- DSCO Router Launches Unified Gateway for Multi-Model Routing with BYOK Support — arthurcolle · 2026-08-24
- Open Source RobotSoul: Persistent Identity for Agents After Context Resets — robauto-dot-ai · 2026-08-24
- Offloading MoE models to RAM causes slow prefill speeds — former_farmer · 2026-08-24
- Etched Raises $1B Led by Jane Street to Validate Architecture-Agnostic AI Chips — TheTuringPost · 2026-08-24
- ConvRot Quant joins llama-cpp: Q6 accuracy nears Q8 quality — giveen · 2026-08-24
- LifeOS: A Local, Voice-Driven Personal Organizer — Extension-Bid-639 · 2026-08-24