Rapid-MLX 0.11.0 brings 25.6x faster first-token latency to Apple Silicon agents
awnihannun · x · 2026-07-26
Rapid-MLX 0.11.0 adds a much faster local inference stack for Apple Silicon and positions it as something you can actually use for agent workflows.
- Prefix-cache reuse cuts a repeated 6k-token prompt from 13.1s to 0.51s TTFT, a 25.6× speedup.
- Response caching drops 656ms to 2ms with byte-identical output and no GPU decode.
- The project says it can hit 152 tok/s on Qwen3.5-4B-4bit, with cold starts in seconds.
- rapid-mlx chat now acts like a real agent: it can connect to any MCP server, chain tools autonomously, and show live tool activity.
- The release also adds support for five model families, including HY3 295B MoE, Qwen3-Coder-Next 80B, MiniCPM5, LFM2.x, and a 2-bit Ternary Bonsai model.
More from coding & agent
- Reticle builds a verification layer that lets AI coding agents check their own generated code — Scobleizer · 2026-10-27
- Trust your agent, but read its logs: Claude managed agents tips — brada · 2026-10-27
- Dev training a guard model to block prompt injection, tool and memory poisoning — gnukeith · 2026-10-27
- Dev trains a ~4GB-VRAM sentinel model to guard against prompt injection and poisoned memory — gnukeith · 2026-10-27
- Superset rebrands as Paradev, raises $11.5M seed led by USV — massimosgrelli · 2026-10-27
- Ex-Big Tech managers may thrive in the AI era: manage agents like you managed teams — lxfater · 2026-10-27