mlx-vlm Multi-Turn Latency Drops to 0.25s
WesEklund · x · 2026-07-12
The author notes their first major contribution to open-source local AI: reducing the multi-turn agent warm latency of mlx-vlm from roughly 72 seconds down to 0.25 seconds, an improvement of about 290x.
Background
- Previously, prefix caching and KV cache quantization could not be enabled simultaneously.
- This meant users had to choose between "faster multi-turn dialogue" and "lower memory usage to fit the model on a Mac."
The Improvement
- By making the two compatible, mlx-vlm can simultaneously retain long-context reuse capabilities and a smaller memory footprint.
- The author emphasizes that this is a massive unlock for agentic workflows.
Conclusion
- This optimization is geared toward local AI and multi-turn reasoning workflows.
- The author also mentioned it will be included in an upcoming release very soon.
Related event: Cache Fix Reduces mlx-vlm Agent Latency by 290x(4 posts)→
More from coding & agent
- He benchmarks new models by having them build their own copilot — $28 of compute — bucolucas · 2026-10-27
- Founder generates 46 app promo videos in 4 days with Claude, skipping $11,500 in UGC fees — PrajwalTomar_ · 2026-10-27
- Should agents be allowed to change plans mid-task? Reddit debates the drift boundary — wistfulcracker84 · 2026-10-27
- HeyGen + Canva MCP turns slide speaker notes into an avatar presentation video — HeyGen · 2026-10-27
- Local beats cloud: Qwen 3.8 FN handles private student data as well as frontier models — dr_lm · 2026-10-27
- Cloudflare Computer adds agent interruption recovery and approval pauses in v0.7.0 — irvinebroque · 2026-10-27