Tracking 1-bit/Ternary models: Bonsai 27B updates & Maple speed boosts
pmttyji · reddit · 2026-08-19
A roundup of low-bit model updates:
- Bonsai: 27B CUDA/Vulkan support merged to llama.cpp mainline; CUDA optimizations yield +15-40% tok/s.
- Maple-Preview: 20B-A1B hits 200+ t/s on Mac Mini M4.
- Mach-1: 35B model achieves 120 t/s on consumer laptops.
- Also tracks Neutrino-8B, Pestle-27B (medical), and various image model quantizations.
More from Infra
- Phison CEO calls CXMT memory scare a 'ghost story', warns of materials shortage — BenBajarin · 2026-10-16
- Ben Bajarin: The Future of Compute Is Fungible as Hyperscalers Design for Workload Flexibility — BenBajarin · 2026-10-16
- AirLLM runs frontier-size models layer-by-layer: 671B on 12GB VRAM — Aiden_Tech_Ai · 2026-10-16
- NVIDIA Launches cuPotential in CUDA-X to Bring AI Agents to Semiconductor Materials Simulation — tomaszbednarz · 2026-10-16
- Q4-quantized coder model on 8-core CPU builds interactive solar system demo in one shot — IntrepidMindExplorer · 2026-10-16
- TokenSpeed MLA 2 ships for Vera Rubin: up to 3.44× faster Kimi K3 decode — zhyncs42 · 2026-10-16