Bonsai 2 challenge: Qwen 27B compressed 10x already 140% faster on Mac, contest open
gajesh · x · 2026-09-26
The MLX.fast community launched a one-week Ternary Bonsai 2 challenge to optimize PrismML's Bonsai 2 — a near-lossless compression of Qwen 3.8 27b to roughly 10% of the original size, fitting on 16GB Apple devices and some phones.
- The community doubled Bonsai's speed within 12 hours; plenty of headroom remains
- Current record: 140.7% speedup over baseline on an M5 Mac (158.2 decode tok/s, 953.5 prefill tok/s) across 17 promoted submissions from 7 solvers
- Score = prefill^0.25 · decode^0.75, so decoding carries 75% of the weight
- Contest runs until October 1 and is open to everyone
More from Infra
- Altman says OpenAI aims to replace Nvidia in the medium to long term — zephyr_z9 · 2026-11-11
- Chinese University Bought Nvidia H200 Chips Despite US Blacklist — pstAsiatech · 2026-11-11
- Baidu's ERNIE Evolve tops public bests on NanoGPT Speedrun, brings RSI to real inference optimization — xiaohu · 2026-11-11
- Baidu says RSI auto-optimized ERNIE inference, 6.24x decode speedup on Kunxin P800 — xiaohu · 2026-11-11
- Nvidia's supply commitments jump from $119B to $279B in one quarter, locking up 2027-28 memory — tengyanAI · 2026-11-11
- Train 27B models on a 24GB GPU: ternary optimizer cuts memory to 1 byte/weight — oguzeldereli · 2026-11-11