Mirai's uzu hits 123 tok/s decode on Apple M5 Max 128GB, fastest for Qwen-3.8 27B
teortaxesTex · x · 2026-10-20
Mirai released a new version of its open-source high-performance inference engine uzu (trymirai/uzu) and benchmarked Qwen-3.8 27B on Apple M5 Max 128GB: 123 tok/s output (with speculative decoding) vs Splash 104, MTPLX 71, oMLX 62, MLX 33, llama.cpp 31; input throughput 1668 tok/s, also fastest. Its new Mirai quantization checkpoints reportedly match Unsloth's quality frontier and clearly beat MLX, shown via a KL-divergence vs size Pareto plot.
More from Infra
- Shadeform launches six features as agent-era compute breaks existing rental models — omarsar0 · 2026-10-20
- Alex Atallah's CAP Theorem for Inference Providers: Reliability, Cost, Speed — AccBalanced · 2026-10-20
- Maxwell: a model that rewrites running binaries to speed up Whisper and LTX-2, with formal verification — EvilHackFar · 2026-10-20
- New paper shows whitening-based inner optimizers enable strong replica scaling in DiLoCo LLM pretraining — kastnerkyle · 2026-10-20
- Together Launches Post-Training Service for Custom Models via RL and SFT — togethercompute · 2026-10-20
- Gemini 3.8 Flash missing in Mumbai region, devs weigh Google's data residency churn — Tech_Partner_01 · 2026-10-20