Rapid-MLX 0.11.0 brings 25.6x faster first-token latency to Apple Silicon agents
awnihannun · x · 2026-07-26
Rapid-MLX 0.11.0 adds a much faster local inference stack for Apple Silicon and positions it as something you can actually use for agent workflows.
- Prefix-cache reuse cuts a repeated 6k-token prompt from 13.1s to 0.51s TTFT, a 25.6× speedup.
- Response caching drops 656ms to 2ms with byte-identical output and no GPU decode.
- The project says it can hit 152 tok/s on Qwen3.5-4B-4bit, with cold starts in seconds.
- rapid-mlx chat now acts like a real agent: it can connect to any MCP server, chain tools autonomously, and show live tool activity.
- The release also adds support for five model families, including HY3 295B MoE, Qwen3-Coder-Next 80B, MiniCPM5, LFM2.x, and a 2-bit Ternary Bonsai model.
More from coding & agent
- Dev builds talk on guardrails workflow for shipping AI-written code without reading it — TejasKumar_ · 2026-09-11
- banteg: Codex auto-review has regressed, blocking steps needed to complete authorized tasks — banteg · 2026-09-11
- lucasmeijer's workflow: handwrite the doc yourself, then have the agent challenge your understanding — lucasmeijer · 2026-09-11
- A doc-anchored agent workflow: you write, the agent only critiques and finds disagreements — lucasmeijer · 2026-09-11
- SymKit MCP: 44 tools for AI agents to verify symbolic derivations — Foreign-Specific-604 · 2026-09-11
- GitHub Copilot team routes user bug reports to an AI agent via Slack — marlene_zw · 2026-09-11