Open-weight 27B model hits ~580 tok/s on Mac, 505% over baseline in 3 days
TheMoonMidas · x · 2026-09-28
PrismML's open-weight Ternary Bonsai 2 27B keeps smashing records in the YukonMLX.fast MLX optimization challenge: decode now reaches 580-600 tok/s on an M5 Mac, 505.4% over the frozen baseline, with decode weighted 75% in the score.
The challenge opened less than 3 days ago and the record has climbed from 4x to over 5x; nearly all top-10 leaderboard entries were built by agents powered by Claude, GPT, Grok, DeepSeek and Gemini. Open to anyone, submissions close 10/1, with 79 promoted submissions from 21 solvers so far.
Key points:
- A 27B open model fits a Mac Mini with near server-class decode speeds
- Agents optimizing inference is becoming a self-reinforcing frontier paradigm
- Score formula prefill^0.25 · decode^0.75; paired official runs prevent cheating
More from coding & agent
- Baseten's Guide: Evaluate Open Models for Coding Agents on Your Own PRs — baseten · 2026-11-16
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16
- ffmpeg-skill gives coding agents local video editing via 42 FFmpeg tools and MCP — kajisho5 · 2026-11-16
- Agents Now Reply to Slack Messages on Their Own, Resolving a Prod Incident Ghost-to-Ghost — iamrobotbear · 2026-11-16
- Engineer's Interview Prep Method: Use Claude to Find Which Answers Undersell Your AI Work — brandon_galang · 2026-11-16
- DeepSeek open-sources its app frontend "chat-demo", already at 3.9k stars — NielsRogge · 2026-11-16