M3 Ultra Hits 374 tok/s Decoding Running LiquidAI 3B Locally at 16 Concurrent Requests
JosephJacks_ · x · 2026-08-13
Nativ, an open-source local model runner, demonstrated the concurrent performance of running LiquidAI's LFM2.5-VL-3B multimodal model on Apple Silicon (M3 Ultra).
- High-Concurrency Decoding: Achieved 374.6 tok/s aggregate decode at 16 concurrent requests, which is 4.16× faster than a single request.
- Fast Prefill: Reached 5,494 tok/s prefill, 3.06× that of a single request.
- Fully Local: No cloud or API bills required; all data stays on the Mac. Available now in Nativ v0.3.1.
More from Infra
- Tencent squeezes 770B-parameter Hunyuan Hy4 Preview into 214 GiB at ~2.38 bits per weight — coolmum18 · 2026-10-16
- TokenSpeed MLA 2 hits NVIDIA Vera Rubin: up to 3.44x faster decode vs B200 — zhyncs42 · 2026-10-16
- US vs Chinese hyperscalers: Americans fund and schmooze, Chinese demand your BoM and cap you at 7% margin — jwt0625 · 2026-10-16
- How LiteLLM routes one OpenAI-shaped API across every model provider — Mahmoud_Zalt · 2026-10-16
- OpenAI-Broadcom custom AI chip Jalapeño named a TIME Best Invention of 2026 — itsclivetime · 2026-10-16
- Developer live-runs continual pretraining of a Luganda LLM on Colab, 41% done with loss dropping fast — weskambale · 2026-10-16