Optimized ANEMLL Fork Runs Qwen3.8-27B on Mac mini M6 Neural Engine, Up to 24.7% Faster
Striking_Luck_886 · reddit · 2026-11-09
A developer bought several 32GB Mac mini M6 machines as local inference hardware ahead of upcoming Qwen4 MoE releases, tested five inference stacks (OMLX, LMLX, Splash, ANEMLL), and built a hyper-optimized ANEMLL fork to run Qwen3.8-27B on the Apple Neural Engine.
Result: 4.4–6.1% faster than original ANEMLL and 2.7–24.7% faster than Splash, via tricks like prompt-lookup drafts, head-free prefill blocks, and faster host decode. The author notes q3 quantization hurts quality and is training a Q4 version that fits. Open source on GitHub.
More from Infra
- Batch APIs: the 50% LLM discount only pays off after model choice and prompt caching — arpit_bhayani · 2026-11-13
- RTX Pro 6000 bricks mid-Qwen run, owner jokes it "died of a broken heart" over Jensen news — wine_dark · 2026-11-13
- Bittensor's Chutes Handles 256 Concurrent Requests on a Single RTX 5090 at ~18k tok/s — markjeffrey · 2026-11-13
- Google expands TPU v8 lineup: 8tx cuts latency 2x, 8i Max doubles silicon per rack — Beth_Kindig · 2026-11-13
- Horace He to share two mental models for reasoning about inference at SGLang Summit — cHHillee · 2026-11-13
- Used A100 prices fall another 20-30% as sellers rush ahead of a collapse — Bascilian · 2026-11-13