MoE Inference Analysis: Qwen 27B vs. Flash-Next 288B on M5 Pro
EyalToledano · x · 2026-08-29
Author compares the inference performance of Qwen 2.5 27B (dense) and Flash-Next REAP-288B (MoE) on an M5 Pro 64GB device.
Core Mechanism:
- Qwen 2.5 27B (Dense): Reads all 27B weights for every single token.
- Flash-Next 288B (MoE): Total 288B weights, but only activates 6B parameters per token.
Performance on M5 Pro (307 GB/s Bandwidth):
- Qwen 2.5 27B @ 4-bit: Uses 18GB VRAM. Reads 16GB/token. Bare decode ceiling is 19 tok/s (prob. 15), potentially reaching 35-50 tok/s with a good MTP drafter.
- Flash-Next REAP-288B @ 4-bit: Reads only 2.3GB/token (7x less than Qwen). Bandwidth is not the bottleneck. Estimated 15-20 tok/s bare, 25-40 tok/s with MTP drafter.
Takeaway: MoE models decode much faster than dense models of equivalent total size, while being significantly stronger than much smaller dense models.
More from Infra
- AI datacenters face visceral physical backlash as expansion meets local resistance — AccBalanced · 2026-08-29
- Running Qwen 3.8 MoE on a single DGX Spark: A practical recipe — QuixiAI · 2026-08-29
- Qwen 3.8 Flash NVFP4 deployment config tested on single DGX — QuixiAI · 2026-08-29
- Kyndryl and Broadcom bet on private AI clouds, with certified talent as the key — DavidLinthicum · 2026-08-29
- Running Qwen3.8-Flash-Next FP8 at 524K context on dual RTX 6000 — SpendLucky1273 · 2026-08-29
- DLSS 5 leaks as modders port Nvidia's AI rendering to Control, Skyrim, and GTA — tomwarren · 2026-08-29