Rapid-MLX 0.11.0 brings 25.6x faster first-token latency to Apple Silicon agents
awnihannun · x · 2026-07-26
Rapid-MLX 0.11.0 adds a much faster local inference stack for Apple Silicon and positions it as something you can actually use for agent workflows.
- Prefix-cache reuse cuts a repeated 6k-token prompt from 13.1s to 0.51s TTFT, a 25.6× speedup.
- Response caching drops 656ms to 2ms with byte-identical output and no GPU decode.
- The project says it can hit 152 tok/s on Qwen3.5-4B-4bit, with cold starts in seconds.
- rapid-mlx chat now acts like a real agent: it can connect to any MCP server, chain tools autonomously, and show live tool activity.
- The release also adds support for five model families, including HY3 295B MoE, Qwen3-Coder-Next 80B, MiniCPM5, LFM2.x, and a 2-bit Ternary Bonsai model.
More from coding & agent
- Shopify design lead on AI-era workflow: designers are becoming full-stack editors — KlausCodes · 2026-10-26
- Ultrametric open-sources Open Startup Repo to let AI agents run startup operations — garrytan · 2026-10-26
- MCP fault injection study: agent task scores drop 60% under realistic tool failures, scale doesn't help — dair_ai · 2026-10-26
- General agents burn ~140M tokens/month on LinkedIn outreach; scripts do it 60x cheaper — ChrisHarpon2 · 2026-10-26
- OpenAI upgrades Decisions API: 400 questions per call, native voice and image support — OpenAIDevs · 2026-10-26
- NLI alone rejects 5/6 new agent memories; a 0.45 similarity gate fixes it all — Alternative_Pin9598 · 2026-10-26