M4 Max multi-agent local inference: capping concurrency at 1 is fastest
Viktri1 · reddit · 2026-10-22
A developer running Qwen 3.8 Flash Next (oq4e-mtp) via oMLX on an M4 Max hit a wall with multi-agent workloads: a single task runs at 50–60 t/s, but throughput craters with concurrency. The best workaround so far is setting Max Concurrent Requests to 1, which works in most scenarios where the KV cache reduces prefill.
When agents each prefill heavily and cache misses dominate, speed drops to 20 t/s and screenshots show the GPU far from saturated. The author asks the community what settings could restore throughput and utilize the GPU fully under concurrency.
More from coding & agent
- Learning robotics for free: Claude Opus 5.5 + the Colorfigures skill explained — CatAstro_Piyush · 2026-10-22
- AI-Agent-Built Encyclopedia Scio Hits 12,426 Articles in 56 Days, Outpacing Early Wikipedia — evisoft · 2026-10-22
- "Claude Duet Sounds Great, But Claude Refuses Test Accounts on Localhost" — DavidKPiano · 2026-10-22
- Amazon blocks Meta's Muse AI agent from shopping on its platform — emmanuelvivier · 2026-10-22
- Microsoft's Guardian Agents treat enterprise AI agents as insider threats — emmanuelvivier · 2026-10-22
- Tencent Hunyuan's WorkForge builds 16.7K verifiable real-world environments for work agents — TencentHunyuan · 2026-10-22