AI agents rewrite inference engine, boosting a 27B model from 66 to 580 tok/s on Mac
a300a300 · reddit · 2026-09-28
According to the post, AI agents (mostly Opus 5.5) rewrote a 27B model's inference engine over three days, speeding it up from 66 tok/s to 580 tok/s (9x) on a Mac via MLX. Details at yukon.org/mlxfast. If it holds up, this is a striking case of AI optimizing its own inference infrastructure, though specifics remain to be verified.
More from coding & agent
- Baseten's Guide: Evaluate Open Models for Coding Agents on Your Own PRs — baseten · 2026-11-16
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16
- ffmpeg-skill gives coding agents local video editing via 42 FFmpeg tools and MCP — kajisho5 · 2026-11-16
- Agents Now Reply to Slack Messages on Their Own, Resolving a Prod Incident Ghost-to-Ghost — iamrobotbear · 2026-11-16
- Engineer's Interview Prep Method: Use Claude to Find Which Answers Undersell Your AI Work — brandon_galang · 2026-11-16
- DeepSeek open-sources its app frontend "chat-demo", already at 3.9k stars — NielsRogge · 2026-11-16