Qwen3.8-9B MLX port runs on 16GB Macs with fast speed
alexcovo_eth · x · 2026-08-18
The 4-bit MLX version of Qwen3.8-9B Abliterated is now available, optimized for Apple Silicon. Benchmarks on M5 Max show 1028 tok/s prefill and 44.6 tok/s generation with 13.05 GB peak memory, making it runnable on 16GB Macs. It is a third-party distillation of Qwen3.5-9B with safety refusals removed.
More from Infra
- First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16